Interface LmProvider

Defines the structure and behavior of an LM Provider.

LmProvider

interface LmProvider {
    abort: (() => Promise<void>);
    api: {
        addHeader: ((key, val) => void);
        csrfToken: (() => null | string);
        del: (<T_4>(uri, verbose?) => Promise<ApiResponse<T_4>>);
        get: (<T>(uri, verbose?) => Promise<ApiResponse<T>>);
        hasCsrfCookie: (() => boolean);
        onResponse: ((hook) => void);
        patch: (<T_3>(uri, payload, verbose?) => Promise<ApiResponse<T_3>>);
        post: (<T_1>(uri, payload, multipart?, verbose?) => Promise<ApiResponse<T_1>>);
        postSse: (<T_5>(uri, payload, onChunk, abortController, parseJson?, multipart?, verbose?, debug?) => Promise<void>);
        put: (<T_2>(uri, payload, verbose?) => Promise<ApiResponse<T_2>>);
        removeHeader: ((key) => void);
        setCsrfToken: ((token) => void);
        setCsrfTokenFromCookie: ((verbose?) => boolean);
    };
    apiKey: string;
    defaults?: LmDefaults;
    infer: ((prompt, params, parseJson?, parseJsonFunc?) => Promise<InferenceResult>);
    info: (() => Promise<Record<string, any>>);
    loadModel: ((name, ctx?) => Promise<void>);
    model: ModelConf;
    models: ModelConf[];
    modelsInfo: (() => Promise<void>);
    name: string;
    onEndEmit?: ((result) => void);
    onError?: ((err) => void);
    onStartEmit?: ((data) => void);
    onToken?: ((t) => void);
    serverUrl: string;
}

Properties

abort api apiKey defaults? infer info loadModel model models modelsInfo name onEndEmit? onError? onStartEmit? onToken? serverUrl

Properties

abort

abort: (() => Promise<void>)

Aborts a currently running inference task.

Type declaration

    • (): Promise<void>

    • Returns Promise<void>

api

api: {
    addHeader: ((key, val) => void);
    csrfToken: (() => null | string);
    del: (<T_4>(uri, verbose?) => Promise<ApiResponse<T_4>>);
    get: (<T>(uri, verbose?) => Promise<ApiResponse<T>>);
    hasCsrfCookie: (() => boolean);
    onResponse: ((hook) => void);
    patch: (<T_3>(uri, payload, verbose?) => Promise<ApiResponse<T_3>>);
    post: (<T_1>(uri, payload, multipart?, verbose?) => Promise<ApiResponse<T_1>>);
    postSse: (<T_5>(uri, payload, onChunk, abortController, parseJson?, multipart?, verbose?, debug?) => Promise<void>);
    put: (<T_2>(uri, payload, verbose?) => Promise<ApiResponse<T_2>>);
    removeHeader: ((key) => void);
    setCsrfToken: ((token) => void);
    setCsrfTokenFromCookie: ((verbose?) => boolean);
}

API utility being used.

Type declaration

  • addHeader: ((key, val) => void)
      • (key, val): void

      • Parameters

        • key: string
        • val: string

        Returns void

  • csrfToken: (() => null | string)
      • (): null | string

      • Returns null | string

  • del: (<T_4>(uri, verbose?) => Promise<ApiResponse<T_4>>)
      • <T_4>(uri, verbose?): Promise<ApiResponse<T_4>>

      • Type Parameters

        • T_4

        Parameters

        • uri: string
        • Optional verbose: boolean

        Returns Promise<ApiResponse<T_4>>

  • get: (<T>(uri, verbose?) => Promise<ApiResponse<T>>)
      • <T>(uri, verbose?): Promise<ApiResponse<T>>

      • Type Parameters

        • T

        Parameters

        • uri: string
        • Optional verbose: boolean

        Returns Promise<ApiResponse<T>>

  • hasCsrfCookie: (() => boolean)
      • (): boolean

      • Returns boolean

  • onResponse: ((hook) => void)
      • (hook): void

      • Parameters

        • hook: OnResponseHook

        Returns void

  • patch: (<T_3>(uri, payload, verbose?) => Promise<ApiResponse<T_3>>)
      • <T_3>(uri, payload, verbose?): Promise<ApiResponse<T_3>>

      • Type Parameters

        • T_3

        Parameters

        • uri: string
        • payload: any[] | Record<string, any>
        • Optional verbose: boolean

        Returns Promise<ApiResponse<T_3>>

  • post: (<T_1>(uri, payload, multipart?, verbose?) => Promise<ApiResponse<T_1>>)
      • <T_1>(uri, payload, multipart?, verbose?): Promise<ApiResponse<T_1>>

      • Type Parameters

        • T_1

        Parameters

        • uri: string
        • payload: any[] | FormData | Record<string, any>
        • Optional multipart: boolean
        • Optional verbose: boolean

        Returns Promise<ApiResponse<T_1>>

  • postSse: (<T_5>(uri, payload, onChunk, abortController, parseJson?, multipart?, verbose?, debug?) => Promise<void>)
      • <T_5>(uri, payload, onChunk, abortController, parseJson?, multipart?, verbose?, debug?): Promise<void>

      • Type Parameters

        • T_5

        Parameters

        • uri: string
        • payload: any[] | FormData | Record<string, any>
        • onChunk: ((payload) => void)
            • (payload): void

            • Parameters

              Returns void

        • abortController: AbortController
        • Optional parseJson: boolean
        • Optional multipart: boolean
        • Optional verbose: boolean
        • Optional debug: boolean

        Returns Promise<void>

  • put: (<T_2>(uri, payload, verbose?) => Promise<ApiResponse<T_2>>)
      • <T_2>(uri, payload, verbose?): Promise<ApiResponse<T_2>>

      • Type Parameters

        • T_2

        Parameters

        • uri: string
        • payload: any[] | Record<string, any>
        • Optional verbose: boolean

        Returns Promise<ApiResponse<T_2>>

  • removeHeader: ((key) => void)
      • (key): void

      • Parameters

        • key: string

        Returns void

  • setCsrfToken: ((token) => void)
      • (token): void

      • Parameters

        • token: string

        Returns void

  • setCsrfTokenFromCookie: ((verbose?) => boolean)
      • (verbose?): boolean

      • Parameters

        • Optional verbose: boolean

        Returns boolean

apiKey

apiKey: string

The key used for authentication with the provider's API.

Optional defaults

defaults?: LmDefaults

Default settings for this provider.

infer

infer: ((prompt, params, parseJson?, parseJsonFunc?) => Promise<InferenceResult>)

Makes an inference based on provided prompt and parameters.

Type declaration

    • (prompt, params, parseJson?, parseJsonFunc?): Promise<InferenceResult>

    • Parameters

      • prompt: string
      • params: InferenceParams
      • Optional parseJson: boolean
      • Optional parseJsonFunc: ((data) => Record<string, any>)
          • (data): Record<string, any>

          • Parameters

            • data: string

            Returns Record<string, any>

      Returns Promise<InferenceResult>

info

info: (() => Promise<Record<string, any>>)

Retrieves information about available server config.

Type declaration

    • (): Promise<Record<string, any>>

    • Returns Promise<Record<string, any>>

loadModel

loadModel: ((name, ctx?) => Promise<void>)

Loads a model by name, with optional context and template.

Type declaration

    • (name, ctx?): Promise<void>

    • Parameters

      • name: string
      • Optional ctx: number

      Returns Promise<void>

model

model: ModelConf

Active model configuration.

models

models: ModelConf[]

List of available model configurations.

modelsInfo

modelsInfo: (() => Promise<void>)

Retrieves information about available models.

Type declaration

    • (): Promise<void>

    • Returns Promise<void>

name

name: string

Identifier for the LM provider.

Optional onEndEmit

onEndEmit?: ((result) => void)

Callback triggered when inference ends.

Type declaration

Optional onError

onError?: ((err) => void)

Callback triggered on errors during inference.

Type declaration

    • (err): void

    • Parameters

      • err: string

      Returns void

Optional onStartEmit

onStartEmit?: ((data) => void)

Callback triggered when inference starts.

Type declaration

Optional onToken

onToken?: ((t) => void)

Callback when a new token is received (typically for authentication).

Type declaration

    • (t): void

    • Parameters

      • t: string

      Returns void

serverUrl

serverUrl: string

The URL endpoint for the provider's server.

Settings

Member Visibility

Theme

On This Page

abortapiapiKeydefaultsinferinfoloadModelmodelmodelsmodelsInfonameonEndEmitonErroronStartEmitonTokenserverUrl