LocalLm types documentation
    Preparing search index...

    Interface InferenceStats

    Represents the statistics of an inference.

    InferenceStats

    const inferenceStats: InferenceStats = {
      ingestionTime: 150,
      inferenceTime: 300,
      totalTime: 450,
      ingestionTimeSeconds: 0.15,
      inferenceTimeSeconds: 0.3,
      totalTimeSeconds: 0.45,
      totalTokens: 200,
      tokensPerSecond: 444
    };
    interface InferenceStats {
        inferenceTime: number;
        inferenceTimeSeconds: number;
        ingestionTime: number;
        ingestionTimeSeconds: number;
        tokensPerSecond: number;
        totalTime: number;
        totalTimeSeconds: number;
        totalTokens: number;
    }

    Hierarchy (View Summary)

    Index

    Properties

    inferenceTime inferenceTimeSeconds ingestionTime ingestionTimeSeconds tokensPerSecond totalTime totalTimeSeconds totalTokens

    Properties

    inferenceTime: number

    The time taken to perform the inference in milliseconds.

    inferenceTimeSeconds: number

    The time taken to perform the inference in seconds.

    ingestionTime: number

    The time taken to ingest the input data in milliseconds.

    ingestionTimeSeconds: number

    The time taken to ingest the input data in seconds.

    tokensPerSecond: number

    The number of tokens processed per second.

    totalTime: number

    The total time taken to perform the inference in milliseconds.

    totalTimeSeconds: number

    The total time taken to perform the inference in seconds.

    totalTokens: number

    The total number of tokens processed.

    Settings

    Member Visibility

    On This Page

    Properties