LocalLm api documentation
    Preparing search index...

    Class Lm

    Represents a Language Model (LM) which acts as a high-level interface to various underlying LM providers.

    Implements

    • LmProvider
    Index

    Constructors

    constructor

    Properties

    abort api apiKey defaults? infer loadModel model modelInfo models modelsInfo name onEndEmit? onError? onStartEmit? onToken? provider providerType serverUrl unloadModel

    Constructors

    • Constructs a new LM instance with the specified provider and parameters.

      Parameters

      • params: LmParams

        The parameters for initializing the LM.

      Returns Lm

      Throws an error if an unknown provider type is specified or if required parameters for a provider are missing.

    Properties

    abort: () => Promise<void>
    api: {
        addHeader: (key: string, val: string) => void;
        csrfToken: () => string | null;
        del: <T>(uri: string, verbose?: boolean) => Promise<ApiResponse<T>>;
        get: <T>(uri: string, verbose?: boolean) => Promise<ApiResponse<T>>;
        hasCsrfCookie: () => boolean;
        onResponse: (hook: OnResponseHook) => void;
        patch: <T>(
            uri: string,
            payload: any[] | Record<string, any>,
            verbose?: boolean,
        ) => Promise<ApiResponse<T>>;
        post: <T>(
            uri: string,
            payload: any[] | Record<string, any> | FormData,
            multipart?: boolean,
            verbose?: boolean,
        ) => Promise<ApiResponse<T>>;
        postSse: <T>(
            uri: string,
            payload: any[] | Record<string, any> | FormData,
            onChunk: (payload: T) => void,
            abortController: AbortController,
            parseJson?: boolean,
            multipart?: boolean,
            verbose?: boolean,
            debug?: boolean,
        ) => Promise<void>;
        put: <T>(
            uri: string,
            payload: any[] | Record<string, any>,
            verbose?: boolean,
        ) => Promise<ApiResponse<T>>;
        removeHeader: (key: string) => void;
        setCsrfToken: (token: string) => void;
        setCsrfTokenFromCookie: (verbose?: boolean) => boolean;
    }
    apiKey: string
    defaults?: LmDefaults
    infer: (
        prompt: string,
        params: InferenceParams,
        options?: InferenceOptions,
    ) => Promise<InferenceResult>

    Makes an inference based on provided prompt and parameters.

    Type Declaration

      • (
            prompt: string,
            params: InferenceParams,
            options?: InferenceOptions,
        ): Promise<InferenceResult>

      • Parameters

        • prompt: string

          The input text for the model to generate a response from.

        • params: InferenceParams

          Parameters that control the behavior of the inference process.

        • Optionaloptions: InferenceOptions

          Some options for the inference query

        Returns Promise<InferenceResult>

        The result of the inference process.

    loadModel: (
        name: string,
        ctx?: number,
        urls?: string | string[],
        onLoadProgress?: OnLoadProgress,
    ) => Promise<void>
    model: ModelConf = ...
    modelInfo: () => Promise<ModelConf<Record<string, any>>>
    models: ModelConf<Record<string, any>>[] = ...
    modelsInfo: () => Promise<ModelConf<Record<string, any>>[]>
    name: string
    onEndEmit?: (result: InferenceResult) => void
    onError?: (err: string) => void
    onStartEmit?: (data: IngestionStats) => void
    onToken?: (t: string) => void
    provider: LmProvider
    providerType: LmProviderType
    serverUrl: string
    unloadModel: (name: string) => Promise<void>

    Settings

    Member Visibility

    On This Page

    Constructors
    Properties