Local Embeddings for Microsoft.Extensions.AI

Microsoft.Extensions.AI gave .NET a single abstraction for AI services: write against IEmbeddingGenerator<string, Embedding<float>> and swap providers without touching your code.

Then you go looking for a provider that runs locally, and the options thin out fast. OpenAI and Azure OpenAI are HTTP calls to someone else's GPU. Ollama is local, but "local" there means installing a separate daemon, keeping it running, and talking to it over http://localhost:11434.

Kjarni is a NuGet package. That's the whole install.

using Kjarni;
using Kjarni.Extensions.AI;
using Microsoft.Extensions.AI;

using var embedder = new Embedder("minilm-l6-v2");
IEmbeddingGenerator<string, Embedding<float>> generator = embedder.AsEmbeddingGenerator();

var result = await generator.GenerateAsync(["Kjarni runs locally with no Python."]);
Console.WriteLine(result[0].Vector.Length);   // 384

No API key. No daemon. No pip install. The model downloads once to a local cache and everything after that runs on your CPU, offline.

Install

dotnet add package Kjarni.Extensions.AI

That pulls in Kjarni (the engine and native library) and Microsoft.Extensions.AI.Abstractions. Native binaries for linux-x64, linux-arm64, win-x64 and osx-arm64 ship inside the package and are selected automatically by RID — there is no second install step and nothing to configure.

Dependency injection

Real apps don't new up an embedder. Register it once:

using Kjarni.Extensions.AI;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddKjarniEmbeddingGenerator(model: "minilm-l6-v2");

var app = builder.Build();

Then inject the standard interface anywhere:

app.MapPost("/embed", async (
    string text,
    IEmbeddingGenerator<string, Embedding<float>> generator) =>
{
    var result = await generator.GenerateAsync([text]);
    return Results.Ok(result[0].Vector.ToArray());
});

Note what your handler depends on: IEmbeddingGenerator<string, Embedding<float>>. Nothing in that signature mentions Kjarni. Swap to Azure OpenAI later and the handler is unchanged.

AddKjarniEmbeddingGenerator registers a singleton, deliberately. Model weights load once and are expensive to re-initialize, and the generator serializes concurrent calls internally, so one instance shared across requests is both safe and correct.

It takes the options you'd expect:

builder.Services.AddKjarniEmbeddingGenerator(
    model: "mpnet-base-v2",   // any embedding model below
    device: "cpu",            // or "gpu"
    cacheDir: null,           // null = default cache location
    normalize: true,          // L2-normalize the output vectors
    quiet: true);             // suppress download progress output

Semantic Kernel and the rest of the ecosystem

Because the registration above satisfies IEmbeddingGenerator<string, Embedding<float>>, anything in the .NET AI ecosystem that consumes that interface resolves your local generator from DI — Semantic Kernel components included. You aren't integrating with Kjarni; you're integrating with the abstraction, and Kjarni happens to be behind it.

This is also why the middleware story still works. Microsoft.Extensions.AI ships delegating generators for caching and OpenTelemetry, and they compose over any implementation, including this one.

Why not just use Ollama?

Ollama is a good answer to a different question. It's a model server: a process you install, start, and keep alive, which you then reach over HTTP. That's the right shape when you want one host serving many models to many clients.

It's the wrong shape when you want a library.

The CI row is the one people underestimate. A unit test that embeds text needs no service container, no health-check wait loop, and no mock — it just runs, on the runner, like any other test.

Which model?

Start with minilm-l6-v2. At 22 MB it embeds in milliseconds on a laptop CPU, and for most retrieval work the quality difference against far larger models is smaller than people expect. Move to mpnet-base-v2 if retrieval quality is measurably limiting you, and to nomic-embed-text if your documents are long.

Are the numbers actually right?

This is the question that matters for a local engine, and it deserves a real answer rather than a benchmark chart.

Kjarni's embeddings match PyTorch's. Not "close enough" — the test suite asserts numerical parity against reference outputs, and the same assertions run in CI on every commit. Going through the IEmbeddingGenerator abstraction changes nothing: the vector from generator.GenerateAsync(...) is identical to the one from embedder.Encode(...), element for element.

That matters because an embedding model that's subtly wrong doesn't crash. It silently returns worse search results, and you find out months later when someone asks why retrieval is bad.

FAQ

Does this need Python installed?

No. Kjarni is a Rust engine compiled to a native library. There is no Python, no PyTorch, no ONNX Runtime, and no CUDA toolkit anywhere in the dependency chain.

Does it need a GPU?

No. Everything above runs on CPU. Pass device: "gpu" to use one if it's there.

Does it work offline?

Yes, after the first run. Models download once into a local cache and are reused. In an air-gapped environment, pre-populate the cache and set cacheDir.

Can I use it in a Docker container?

Yes, and the image doesn't need a Python layer, a CUDA base, or a sidecar model server. The native library ships in the NuGet package.

Does it implement IChatClient too?

Not yet. Kjarni's engine supports decoder models and seq2seq generation, but the Microsoft.Extensions.AI surface currently exposes embeddings only. That's the piece most .NET apps need first, and it's the piece where running locally has no real competition.

What does it cost?

Nothing. MIT licensed, no API key, no per-token billing.

Try it

dotnet add package Kjarni.Extensions.AI

Source and issues: github.com/olafurjohannsson/kjarni