Interface ModelState

Represents the state of the available models on server.

ModelState

const modelState: ModelState = {
  models: { gpt3: { name: 'gpt-3', ctx: 2048 } },
  isModelLoaded: true,
  loadedModel: 'gpt-3',
  ctx: 2048
};

interface ModelState {
    ctx: number;
    isModelLoaded: boolean;
    loadedModel: string;
    models: Record<string, ModelTemplate>;
}

Properties

ctx: number

The loaded model context window size, 0 if no model is loaded.

isModelLoaded: boolean

Indicates whether a model is loaded or not.

loadedModel: string

The loaded model name, empty if no model is loaded.

models: Record<string, ModelTemplate>

The models info object (name, template name, context window size).