Interface InferenceStats

Represents the statistics of an inference.

InferenceStats

interface InferenceStats {
    inferenceTime: number;
    inferenceTimeSeconds: number;
    ingestionTime: number;
    ingestionTimeSeconds: number;
    tokensPerSecond: number;
    totalTime: number;
    totalTimeSeconds: number;
    totalTokens: number;
}

Hierarchy (view full)

Properties

inferenceTime inferenceTimeSeconds ingestionTime ingestionTimeSeconds tokensPerSecond totalTime totalTimeSeconds totalTokens

Properties

inferenceTime

inferenceTime: number

The time taken to perform the inference in milliseconds.

inferenceTimeSeconds

inferenceTimeSeconds: number

The time taken to perform the inference in seconds.

ingestionTime

ingestionTime: number

The time taken to ingest the input data in milliseconds.

ingestionTimeSeconds

ingestionTimeSeconds: number

The time taken to ingest the input data in seconds.

tokensPerSecond

tokensPerSecond: number

The number of tokens processed per second.

totalTime

totalTime: number

The total time taken to perform the inference in milliseconds.

totalTimeSeconds

totalTimeSeconds: number

The total time taken to perform the inference in seconds.

totalTokens

totalTokens: number

The total number of tokens processed.

Settings

Member Visibility

Theme

On This Page

inferenceTimeinferenceTimeSecondsingestionTimeingestionTimeSecondstokensPerSecondtotalTimetotalTimeSecondstotalTokens