Skip to content

Store cost data in database #6261

@GabrielBianconi

Description

@GabrielBianconi

Follow up to #6260

The values should be stored in aggregate for the whole inference in ChatInference or JsonInference; and for the individual model inference in ModelInference. We'll need to add cost columns to all those tables in both backends.

Proposed:

  • Postgres: `NUMERIC(18, 9)
    • ClickHouse: Decimal(18, 9)

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority-highThe team is prioritizing this issue and will tackle it very soon.rustThis issue involves Rust.

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions