# A local IChatClient for Microsoft.Extensions.AI

> Implement IChatClient with the model inside your .NET process. No daemon, no localhost port. Streaming, multi-turn and Semantic Kernel.

Source: https://kjarni.ai/blog/ichatclient/ (published 2026-08-27)

`Microsoft.Extensions.AI` gave .NET one abstraction for language models, but the
providers that implement it are mostly HTTP calls to somebody else's GPU. Ollama and
LM Studio are local in the sense that the GPU is yours, but you still install a
daemon, keep it running, and talk to it over `http://localhost:11434`.

This one loads the weights into your process:

```cs
using Kjarni.Extensions.AI;
using Microsoft.Extensions.AI;

using IChatClient client = new KjarniChatClient("llama3.2-3b-instruct");

var response = await client.GetResponseAsync("Explain RAG (retrieval-augmented generation) in one sentence.");
Console.WriteLine(response.Text);
```

```bash
dotnet add package Kjarni.Extensions.AI
```

That pulls in the engine and its native library along with
`Microsoft.Extensions.AI.Abstractions`. Binaries for linux-x64, linux-arm64, win-x64
and osx-arm64 ship inside the package and are picked by RID, so there is no second
install step. Weights download once to a local cache and everything after that is
offline.

## Dependency injection

```cs
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddKjarniChatClient("llama3.2-3b-instruct");
var app = builder.Build();

app.MapPost("/ask", async (string question, IChatClient client) =>
{
    var response = await client.GetResponseAsync(question);
    return Results.Ok(response.Text);
});
```

Nothing about Kjarni appears in that handler's signature. Point it at Azure OpenAI in
production and the handler does not change.

`AddKjarniChatClient` registers a singleton, deliberately: weights load once and are
expensive to re-initialise. That has consequences under load, covered below.

## Multi-turn

`IChatClient` is stateless by design, so you own the transcript and replay it each
call. Kjarni keeps nothing between calls.

```cs
List<ChatMessage> conversation =
[
    new(ChatRole.System, "You are a terse assistant. Answer in one sentence."),
    new(ChatRole.User,   "My name is Olafur and I work on inference engines."),
];

var first = await client.GetResponseAsync(conversation);
conversation.Add(new ChatMessage(ChatRole.Assistant, first.Text));

conversation.Add(new ChatMessage(ChatRole.User, "What do I work on?"));
var second = await client.GetResponseAsync(conversation);

Console.WriteLine(second.Text);   // Inference engines.
```

## Streaming and cancellation

```cs
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5));

await foreach (var update in client.GetStreamingResponseAsync(conversation, null, cts.Token))
    Console.Write(update.Text);
```

Each `ChatResponseUpdate` carries a fragment as the model produces it. Cancellation
stops generation at the next token rather than running to completion and discarding
the result, and the loop then throws `OperationCanceledException`, so catch it where a
cut-off reply is expected.

## Semantic Kernel

Registration is the same, since SK builds on the same abstractions:

```cs
var builder = Kernel.CreateBuilder();
builder.Services.AddKjarniChatClient("llama3.2-3b-instruct");
var kernel = builder.Build();
```

Any SK component resolving `IChatClient` now runs against a local model. Combined
with [local embeddings](/blog/embeddinggenerator/) the whole pipeline stays on your
hardware.

## Options that map, and options that don't

```cs
var options = new ChatOptions
{
    Temperature = 0f,        // greedy decoding
    TopP = 0.95f,
    TopK = 40,
    MaxOutputTokens = 512,
};
```

`Temperature = 0` switches to greedy decoding, so the same prompt returns the same
answer every time. That is what you want for extraction and classification, where a
different answer per call is a bug rather than variety.

The rest is explicit rather than quietly dropped:

| Option | Behaviour |
| ------ | --------- |
| `Tools` | **Throws `NotSupportedException`** |
| `ResponseFormat = Json` | **Throws**, no constrained decoding |
| `ChatRole.Tool` messages | **Throws** |
| `Seed`, `StopSequences` | Ignored |
| `FrequencyPenalty`, `PresencePenalty` | Ignored |

Throwing on `Tools` is a choice. Accepting the option and ignoring it would leave you
believing your functions had been offered to the model when they never were, which is
a bug you find in production rather than at the call site. If you need function
calling, this is not your library today.

`FrequencyPenalty` is ignored rather than mapped onto the repetition penalty, because
the two are different things and substituting one for the other would change your
output in ways you did not ask for.

## Concurrency

Generation is compute bound and the native model is not re-entrant, so calls are
serialised per instance. Two simultaneous requests queue rather than overlap, and a
request waiting on the gate is waiting for a real CPU, not a socket.

So a single instance is not a throughput story. It is good for a desktop app, a CLI,
a background worker, a batch job, or an internal tool with a handful of users. For a
high traffic public endpoint you want a queue in front of it or a hosted provider,
and because you coded against `IChatClient` that is a registration change.

## Wrapping a Chat you already have

```cs
using var chat = new Chat("llama3.2-3b-instruct", quiet: true);
var client = chat.AsChatClient();
```

`AsChatClient` and `AddKjarniChatClient` live in the `Microsoft.Extensions.AI`
namespace, so they appear alongside the abstractions with no third `using` to guess
at. The client borrows the chat by default, so disposing it leaves your `Chat` alive.
Pass `ownsChat: true` to hand over ownership.

## Going underneath the abstraction

```cs
var chat = client.GetService(typeof(Chat)) as Chat;
Console.WriteLine(chat?.ContextSize);        // 131072

var metadata = client.GetService(typeof(ChatClientMetadata)) as ChatClientMetadata;
Console.WriteLine(metadata?.ProviderName);   // kjarni
```

## FAQ

### How do I fix "No service for type 'Microsoft.Extensions.AI.IChatClient' has been registered"?

Register an `IChatClient` before calling `Build()`, for example with
`builder.Services.AddKjarniChatClient("llama3.2-3b-instruct")`. The error means something
asked the container for an `IChatClient` and nothing was registered under that interface.

```
System.InvalidOperationException: No service for type 'Microsoft.Extensions.AI.IChatClient' has been registered.
```

The request can come from a constructor parameter, a minimal API handler or a
`GetRequiredService<IChatClient>()` call. The fix is the same for all of them:

```cs
builder.Services.AddKjarniChatClient("llama3.2-3b-instruct");
```

Two other ways to get the same error:

- **Registering the concrete type.** `services.AddSingleton(new KjarniChatClient(...))`
  registers a `KjarniChatClient`, not an `IChatClient`, and the container does not map one
  onto the other. Use `AddKjarniChatClient`, or
  `services.AddSingleton<IChatClient>(new KjarniChatClient(...))`.
- **Registering in a different container.** A kernel from `Kernel.CreateBuilder()` has
  its own services, separate from the web host's. When the kernel resolves the client,
  register it on the kernel builder's `Services`, as in the Semantic Kernel section above.

### How do I set the temperature on an IChatClient?

Pass `ChatOptions` with `Temperature` set on each call, or set a default for every call
with `ConfigureOptions`. Without either, Kjarni samples at 0.7.

```cs
var options = new ChatOptions { Temperature = 0f, MaxOutputTokens = 200 };
var response = await client.GetResponseAsync("Name three fruits, comma separated.", options);
```

For every call, wrap the client with `ConfigureOptions` from the `Microsoft.Extensions.AI`
package (`dotnet add package Microsoft.Extensions.AI`):

```cs
builder.Services.AddChatClient(_ => new KjarniChatClient("llama3.2-3b-instruct"))
                .ConfigureOptions(o => o.Temperature ??= 0f);
```

`??=` keeps a temperature that a caller passes explicitly. At `Temperature = 0` the same prompt gives the same reply: asked for three
fruits twice, llama3.2-3b-instruct answered "Apple, Banana, Mango" both times.

### Does it support tool calling?

Not yet. `ChatOptions.Tools` throws rather than silently ignoring your functions. The
engine supports it in principle, the surface does not expose it.

### Does it implement IEmbeddingGenerator too?

Yes, from the same package, so a Semantic Kernel RAG pipeline runs entirely locally. See
[Local Embeddings for Microsoft.Extensions.AI](/blog/embeddinggenerator/).

## More

For model choice, hardware expectations and how this compares with running a daemon,
see [Local LLM chat in C#](/blog/localchat/).

```bash
dotnet add package Kjarni.Extensions.AI
```

Source and issues: [github.com/olafurjohannsson/kjarni](https://github.com/olafurjohannsson/kjarni)
