Transformer inference library

Run transformer models inside your program, in any language.

Same output as PyTorch. One native engine for Python, C#, Go, C++, Rust and the browser.

Install

$ pip install kjarni

Then read the docs → Or try it in your browser →

Embeddings in a few lines

The model downloads once, then everything runs offline.

from kjarni import Embedder

e = Embedder("minilm-l6-v2")
print(e.similarity("doctor", "physician"))
# 0.8598

using Kjarni;

using var embedder = new Embedder("minilm-l6-v2");
Console.WriteLine(embedder.Similarity("doctor", "physician"));
// 0.8598

e, _ := kjarni.NewEmbedder("minilm-l6-v2")
defer e.Close()

sim, _ := e.Similarity("doctor", "physician")
fmt.Println(sim)
// 0.8598

let embedder = kjarni::Embedder::new("minilm-l6-v2").await?;
let sim = embedder.similarity("doctor", "physician").await?;
println!("{sim:.4}");
// 0.8598

#include "kjarni.hpp"

auto embedder = kjarni::Embedder::create({.model = "minilm-l6-v2"});
auto a = embedder->encode("doctor");
auto b = embedder->encode("physician");
std::println("{:.4f}", kjarni::cosine(*a, *b));

import { Kjarni } from "kjarni-wasm";

const kjarni = await Kjarni.load({ encoder: url });
console.log(await kjarni.similarity("doctor", "physician"));
// runs in a Web Worker, no server

$ kjarni similarity "doctor" "physician"
  █████████████████░░░   86.0%  highly similar

$ cat reviews.txt | kjarni classify --format json

Same model, same number, in every language.Docs →

Checked against PyTorch

Kjarni compared with Hugging Face transformers on the same weights.

all-MiniLM-L6-v2 embeddings	max diff 1.4e-7
BGE-M3 embeddings	max diff 2e-7
Llama 3.2 3B greedy decode	identical
Phi-3.5 Mini greedy decode	identical
Mistral 7B, Qwen 2.5 greedy decode	identical
Encoders on the GPU every model	max diff 6e-6

Full-precision weights on CPU. Quantized weights trade exactness for speed.Details →

Ready before the others have loaded

Time to load all-MiniLM-L6-v2, in milliseconds (lower is better)

Kjarni

ONNX Runtime

495

PyTorch

2,500

i7-13700, CPU. The gap narrows on larger models: bge-m3 loads in 1.3 s, 1.5 s and 6.7 s.Reproduce →

Runs on Linux x64, arm64 macOS Apple Silicon Windows x64 Browser WebAssembly GPU Vulkan, Metal, DirectX 12

Eight tasks, one engine.
Use one, or all of them.

Every task also runs from the command line, and kjarni mcp serves them to AI agents.

Embeddings

instead of sentence-transformers

How close two texts are in meaning: related items, duplicates, FAQ matching.

$ kjarni embed "Hello"

Classification

instead of transformers

Sentiment, emotion and toxicity, with no training.

$ kjarni classify "Love it!"

Search

instead of a search server

Keyword, semantic or hybrid search over a folder. The index is a directory on disk.

$ kjarni search docs "backups"

Reranking

instead of a rerank API

A cross-encoder rescores your top results so the right one comes first.

$ kjarni rerank "q" < hits.txt

Chat

instead of Ollama or an API key

Llama, Qwen, Phi and Mistral in your process, streaming, on CPU or GPU.

$ kjarni chat

Transcription

instead of openai-whisper

Whisper speech-to-text with timestamps, from wav, mp3, flac or ogg.

$ kjarni transcribe call.mp3

Summarization

instead of a summarization API

BART or FLAN-T5 turn a long document into a short one.

$ kjarni summarize < doc.txt

Translation

instead of a translation API

For text that is not allowed to leave your machine.

$ kjarni translate --dst de

Not every task is in every language yet. See what each language supports →

A minute with Kjarni

See it run.

Every step is a real command with its real output and timing, recorded on a desktop CPU with no GPU. Click one, or just watch.

Kjarni 0.4.1 on Linux x86-64, models already downloaded. Hover to pause.

One install, in the language you already use.

Install Read the docs