LocalLm api documentation
    Preparing search index...

    Class LlamacppProvider

    Implements

    • LmProvider
    Index

    Constructors

    constructor

    Properties

    abortController api apiKey model models name onEndEmit? onError? onStartEmit? onToken? serverUrl

    Methods

    abort infer loadModel modelInfo modelsInfo unloadModel

    Constructors

    Properties

    abortController: AbortController = ...
    api: {
        addHeader: (key: string, val: string) => void;
        csrfToken: () => string | null;
        del: <T>(uri: string, verbose?: boolean) => Promise<ApiResponse<T>>;
        get: <T>(uri: string, verbose?: boolean) => Promise<ApiResponse<T>>;
        hasCsrfCookie: () => boolean;
        onResponse: (hook: OnResponseHook) => void;
        patch: <T>(
            uri: string,
            payload: any[] | Record<string, any>,
            verbose?: boolean,
        ) => Promise<ApiResponse<T>>;
        post: <T>(
            uri: string,
            payload: any[] | Record<string, any> | FormData,
            multipart?: boolean,
            verbose?: boolean,
        ) => Promise<ApiResponse<T>>;
        postSse: <T>(
            uri: string,
            payload: any[] | Record<string, any> | FormData,
            onChunk: (payload: T) => void,
            abortController: AbortController,
            parseJson?: boolean,
            multipart?: boolean,
            verbose?: boolean,
            debug?: boolean,
        ) => Promise<void>;
        put: <T>(
            uri: string,
            payload: any[] | Record<string, any>,
            verbose?: boolean,
        ) => Promise<ApiResponse<T>>;
        removeHeader: (key: string) => void;
        setCsrfToken: (token: string) => void;
        setCsrfTokenFromCookie: (verbose?: boolean) => boolean;
    }
    apiKey: string
    model: ModelConf = ...
    models: ModelConf<ModelData>[] = ...
    name: string
    onEndEmit?: (result: InferenceResult) => void
    onError?: (err: string) => void
    onStartEmit?: (data: IngestionStats) => void
    onToken?: (t: string) => void
    serverUrl: string

    Methods

    • Aborts a currently running inference task.

      Returns Promise<void>

    • Makes an inference based on the provided prompt and parameters.

      Parameters

      • prompt: string

        The input text to base the inference on.

      • params: InferenceParams

        Parameters for customizing the inference behavior.

      • Optionaloptions: InferenceOptions

      Returns Promise<InferenceResult>

      • The result of the inference.

    • Loads a specified model for inference

      Parameters

      • name: string

        The name of the model to load.

      • Optionalctx: number

        The optional context window length, defaults to the model ctx.

      • Optionalurls: string | string[]
      • OptionalonLoadProgress: OnLoadProgress

      Returns Promise<void>

    • Returns Promise<ModelConf<Record<string, any>>>

    • Load the models list

      Returns Promise<ModelConf<ModelData>[]>

    • Unloads a specified model

      Parameters

      • name: string

        The name of the model to unload.

      Returns Promise<void>

    Settings

    Member Visibility

    On This Page

    Constructors
    Properties
    Methods