Documentation
Kjarni documentation: capabilities, languages, platforms and models
What Kjarni can do from each language, on which platforms, with which models, and on what hardware. Embeddings, classification, reranking, keyword/semantic/hybrid search, chat, summarization, translation, speech-to-text and image search, running locally from Rust, C#, Python, Go, C/C++, the browser and the command line.
Kjarni is a local inference engine for transformer models. It is written in Rust and ships as one native library with bindings for C#, Python, Go and C/C++, as a WebAssembly package for the browser, and as a command line tool. Models download from HuggingFace on first use and are cached, so everything after the first run works offline. It needs no Python runtime, no ONNX Runtime and no CUDA toolkit. On Linux the native library links only against glibc.
This page is the reference for what is supported where. It is kept to what the released packages actually expose; anything that exists in the engine but not yet in a binding is marked as such.
What each language can do
| Capability | Rust | CLI | C# | Python | Go | C / C++ | Browser |
|---|---|---|---|---|---|---|---|
| Embeddings and similarity | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Text classification (sentiment, emotion, toxicity) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Reranking (cross-encoder) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Index and search: keyword (BM25), semantic, hybrid | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ ¹ | ✓ ² |
| Chat with a local LLM, streaming | ✓ | ✓ | ✓ | – | ✓ | ✓ | ✓ |
| Text generation (GPT-2 family) | ✓ | ✓ | – | – | – | – | – |
| Summarization | ✓ | ✓ | – | – | – | – | – |
| Translation | ✓ | ✓ | – | – | – | – | – |
| Speech-to-text (Whisper) | ✓ | ✓ | – | – | – | – | – |
| Image search by description (CLIP) | ✓ ³ | ✓ | – | – | – | – ⁴ | – |
IChatClient and IEmbeddingGenerator (Microsoft.Extensions.AI) | ✓ ⁵ |
- Through the C API (
kjarni.h). The C++23 wrapperkjarni.hppcovers embeddings, classification, reranking and chat; indexing and search are called through the C functions directly. - In-memory index built in the page, through the lower-level
WasmSearchandWasmIndexBuilderclasses rather than the high-levelKjarniclass. - Behind the
image-iocargo feature. - The C API for image search exists behind a build flag that the released libraries do not enable.
- The
Kjarni.Extensions.AIpackage, .NET 8. Tool calling is not supported yet.
Installing
| Language | Install | Package |
|---|---|---|
| C# / .NET | dotnet add package Kjarni | NuGet |
| Microsoft.Extensions.AI | dotnet add package Kjarni.Extensions.AI | NuGet |
| Python | pip install kjarni | PyPI |
| Go | go get github.com/olafurjohannsson/kjarni-go | pkg.go.dev |
| Rust | cargo add kjarni | crates.io |
| Browser | npm i kjarni-wasm | npm |
| C / C++ | Release archive: shared library, kjarni.h, kjarni.hpp | Releases |
| Command line | curl -fsSL https://kjarni.ai/install.sh | sh | Releases |
On Windows the command line tool installs with irm https://kjarni.ai/install.ps1 | iex.
Kjarni on Windows lists every file in the Windows archive and what it
is for.
Generated API references, built from the source by their registries:
Rust on docs.rs and
Go on pkg.go.dev. For the C
and C++ API, kjarni.h
documents every function, and
kjarni.hpp
is a header-only wrapper over it.
Platforms
| Package | Linux x64 | Linux arm64 | macOS Apple Silicon | Windows x64 | Browser |
|---|---|---|---|---|---|
| Release archives and CLI | ✓ | ✓ | ✓ | ✓ | |
NuGet (Kjarni) | ✓ | ✓ | ✓ | ✓ | |
PyPI (kjarni), Python 3.8+ | ✓ | ✓ | ✓ | ✓ | |
| Go module | ✓ | – | – | ✓ | |
npm (kjarni-wasm) | ✓ |
Not supported yet: macOS on Intel, Windows on ARM, Android and iOS.
The Kjarni NuGet package targets .NET 8 and .NET Standard 2.1. .NET Standard 2.1 covers
Unity 2021.2 and later; it does not cover .NET Framework 4.x, which stops at .NET Standard
2.0. See Local AI in Unity.
Models
The registry below is what kjarni model list shows, and the names are what every binding
accepts. Chat models are also available as GGUF downloads, which are smaller.
| Task | Model | Parameters |
|---|---|---|
| Embeddings | minilm-l6-v2 (default) | 22M |
mpnet-base-v2 | 110M | |
nomic-embed-text | 137M | |
bge-m3 (multilingual) | 567M | |
distilbert-base | 66M | |
| Reranking | minilm-l6-v2-cross-encoder | 22M |
| Classification | distilbert-sentiment (positive/negative) | 66M |
roberta-sentiment (negative/neutral/positive) | 125M | |
bert-sentiment-multilingual (1 to 5 stars) | 168M | |
toxic-bert (six toxicity labels) | 110M | |
roberta-emotions (28 emotions, multi-label) | 125M | |
distilroberta-emotion (7 emotions) | 82M | |
| Chat | qwen2.5-0.5b-instruct | 0.5B |
qwen2.5-1.5b-instruct | 1.5B | |
llama3.2-1b-instruct | 1.2B | |
llama3.2-3b-instruct | 3.2B | |
phi3.5-mini-instruct | 3.8B | |
mistral-7b-instruct | 7.2B | |
llama3.1-8b-instruct | 8.0B | |
deepseek-r1-8b | 8.0B | |
| Text generation | distilgpt2, gpt2 | 82M, 117M |
| Summarization | distilbart-cnn, bart-large-cnn | 306M, 406M |
| Instruction / translation | flan-t5-base, flan-t5-large | 250M, 780M |
| Speech-to-text | whisper-small, whisper-large-v3 | 244M, 1.6B |
| Image and text embeddings | clip-vit-base-32 | 151M |
The engine implements these architecture families: BERT, DistilBERT, RoBERTa, XLM-RoBERTa,
MPNet and Nomic-BERT encoders; Llama, Qwen 2, Phi 3, Mistral and GPT-2 decoders; T5, BART
and Whisper encoder-decoders; and CLIP. A model outside the registry can be loaded from a
local folder (model_path in the binding configs, --model-path on the command line)
provided it uses one of those architectures. Weights load from HuggingFace safetensors or
GGUF as published; there is no conversion step.
Embedding models truncate at the length their sentence-transformers config specifies, the
same as the Python reference: 256 tokens for minilm-l6-v2 and 384 for mpnet-base-v2.
The 256-token limit explains what that means for chunking.
Hardware
Everything runs on CPU by default, using all physical cores within a call, with SIMD kernels for x86-64 (AVX2) and ARM (NEON), and int4/int8 quantized weights for chat models.
GPU support goes through WebGPU (the wgpu library), so it uses Vulkan, DirectX 12 or
Metal and works on AMD, Intel, Apple and NVIDIA hardware without a CUDA toolkit. Pass
gpu: true (or --gpu, or device: "gpu" depending on the binding).
| Model type | GPU status |
|---|---|
| Encoders: embeddings, classification, reranking | Verified against CPU output |
| Decoders: chat and generation | Works; less tested than CPU |
| Encoder-decoders: summarization, translation, Whisper | Less tested; prefer CPU |
In the browser, kjarni-wasm runs on CPU with WebAssembly SIMD, in a Web Worker so the
page does not freeze. A WebGPU build of the engine exists in the repository but is not in
the npm package yet.
Offline use and the model cache
Models download on first use and are cached in ~/.cache/kjarni on Linux and macOS and
%LOCALAPPDATA%\kjarni on Windows. Once a model is cached, no network is used. Set
KJARNI_CACHE_DIR to move the cache, or pass a cache directory per model in the binding.
A pre-filled cache folder can ship with an application, read-only, so it never needs the
network at all. Set HF_TOKEN for gated HuggingFace models.
What Kjarni is not
- Not a training framework. It runs inference only.
- Not a general model runtime. It runs the architecture families above, implemented natively. For an arbitrary research model, ONNX Runtime or PyTorch is the right tool.
- Not a server. It is a library in your process. Nothing listens on a port.
- Not on mobile yet. No Android or iOS builds.
FAQ
Does Kjarni need Python?
No. The engine is a native library, so a C#, Go, C++, Rust or command line program needs no Python installed. Kjarni also publishes Python bindings on PyPI for use from Python.
Does Kjarni need a GPU or CUDA?
No. Everything runs on CPU by default. GPU support is optional and goes through WebGPU, using Vulkan, DirectX 12 or Metal, so no CUDA toolkit is involved.
Does it work offline?
Yes, after a model is downloaded once. Models can also be shipped with an application and loaded from a local folder so the network is never used.
Can I run a local LLM with Kjarni from C#?
Yes. The Chat class in the Kjarni package runs Qwen, Llama, Phi, Mistral and DeepSeek
R1 models in-process with streaming, and Kjarni.Extensions.AI exposes it as an
IChatClient. Tool calling is not supported yet.
Can I do semantic search without a vector database?
Yes. Kjarni's indexer builds a keyword (BM25) and vector index on disk, and the searcher queries it in keyword, semantic or hybrid mode, with optional cross-encoder reranking. This is available from Rust, C#, Python, Go, C/C++ and the command line.
Does Kjarni produce the same embeddings as sentence-transformers?
Yes, for the models in the registry: same weights, same pooling, same truncation length. The C# and C++ posts on this site show the numbers side by side.
Which languages can transcribe audio?
Rust and the command line (kjarni transcribe). Whisper is not exposed in the C#,
Python, Go or C bindings yet.
Does Kjarni run in the browser?
Yes. The kjarni-wasm npm package runs embeddings, classification, reranking, search and
chat in a Web Worker, on CPU with WebAssembly SIMD. The live demo runs three
models in the page.
What license is Kjarni under?
MIT or Apache 2.0, at your option.
Guides by language
- C#: Semantic search, sentiment analysis, document search engine, reranking, RAG without a vector database, local LLM chat, IChatClient, IEmbeddingGenerator
- C++: Semantic search without libtorch or ONNX Runtime, local LLM, Qt Quick search
- Python: Semantic search
- Unity: Local AI in Unity
- Windows: Which file you need
- Command line: ML from the command line
- Background: Why a native engine, benchmarking against PyTorch, the MiniLM 256-token limit
Kjarni runs the same engine on every platform: