Class KjarniChatClient

Namespace: Kjarni.Extensions.AI
Assembly: Kjarni.Extensions.AI.dll

An backed by a local Kjarni . The model is loaded directly by your application, so prompts and responses stay on the machine and there is nothing listening on a port to keep running alongside it.

public sealed class KjarniChatClient : IChatClient, IDisposable

Inheritance

object ← KjarniChatClient

Implements

IChatClient, IDisposable

Remarks

is stateless — the caller replays the whole transcript on every call — so this maps onto rather than . Nothing is retained between calls, which is what a shared singleton in a web application needs.

The underlying native chat is not re-entrant, so concurrent calls are serialized internally. Because generation is compute-bound rather than I/O-bound, a request that waits on the gate is waiting for a real CPU or GPU, not for a socket: throughput is one conversation at a time per instance.

Constructors

KjarniChatClient(string, string?, ChatMode, string, bool)

Creates a client that owns its own .

public KjarniChatClient(string model = "llama3.2-3b-instruct", string? systemPrompt = null, ChatMode mode = ChatMode.Default, string device = "cpu", bool quiet = true)

Parameters

model string

Kjarni model name, e.g. llama3.2-3b-instruct.

systemPrompt string?

Default system prompt, or null for none.

mode ChatMode

Chat mode preset.

device string

"cpu" or "gpu".

quiet bool

Suppress model download/progress output.

KjarniChatClient(Chat, string?, bool)

Wraps an existing .

public KjarniChatClient(Chat chat, string? modelId = null, bool ownsChat = false)

Parameters

chat Chat

The chat instance to delegate to.

modelId string?

Model identifier reported through metadata.

ownsChat bool

When true, disposing this client also disposes chat.

Properties

ContextSize

The context window of the underlying model, in tokens.

public int ContextSize { get; }

Property Value

int

Methods

Dispose()

Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.

public void Dispose()

GetResponseAsync(IEnumerable<ChatMessage>, ChatOptions?, CancellationToken)

Sends chat messages and returns the response.

public Task<ChatResponse> GetResponseAsync(IEnumerable<ChatMessage> messages, ChatOptions? options = null, CancellationToken cancellationToken = default)

Parameters

messages IEnumerable<ChatMessage>

The sequence of chat messages to send.

options ChatOptions?

The chat options with which to configure the request.

cancellationToken CancellationToken

The to monitor for cancellation requests. The default is .

Returns

Task<ChatResponse>

The response messages generated by the client.

Exceptions

ArgumentNullException

messages is null.

GetService(Type, object?)

Asks the for an object of the specified type serviceType.

public object? GetService(Type serviceType, object? serviceKey = null)

Parameters

serviceType Type

The type of object being requested.

serviceKey object?

An optional key that can be used to help identify the target service.

Returns

object?

The found object, otherwise null.

Remarks

The purpose of this method is to allow for the retrieval of strongly typed services that might be provided by the , including itself or any services it might be wrapping. For example, to access the for the instance, may be used to request it.

Exceptions

ArgumentNullException

serviceType is null.

GetStreamingResponseAsync(IEnumerable<ChatMessage>, ChatOptions?, CancellationToken)

Sends chat messages and streams the response.

public IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(IEnumerable<ChatMessage> messages, ChatOptions? options = null, CancellationToken cancellationToken = default)

Parameters

messages IEnumerable<ChatMessage>

The sequence of chat messages to send.

options ChatOptions?

The chat options with which to configure the request.

cancellationToken CancellationToken

The to monitor for cancellation requests. The default is .

Returns

IAsyncEnumerable<ChatResponseUpdate>

The response messages generated by the client.

Remarks

Native generation is a blocking callback loop, so it runs on a background thread and feeds an unbounded channel that this method drains. Cancelling stops generation at the next token rather than after the full response, because the token callback reports the cancellation back to the native side.

Exceptions

ArgumentNullException

messages is null.