Why I Built a Native ML Inference Engine in Rust
Kjarni is a single native library that runs transformer models in your own process, from Rust, C#, Go, C++ or the command line. One install, on CPU, offline after the first run.
Technical articles on AI inference, search, embeddings, and building developer tools.
Kjarni is a single native library that runs transformer models in your own process, from Rust, C#, Go, C++ or the command line. One install, on CPU, offline after the first run.
Run sentiment analysis locally in C# using transformer models. Positive, negative, neutral, emotions, toxicity. Three lines of code, one NuGet package, running on your own machine.
Add semantic search to your C# app in 5 lines. Match text by meaning, not keywords. One NuGet package with zero dependencies, running on CPU in your own process.
Build a full-text search engine in C# with keyword search, semantic search, hybrid ranking, and reranking. Index files and query them in 10 lines of code, with the index living on your own disk.
Run sentiment analysis, generate embeddings, detect toxicity, and search documents from your terminal. One binary, reads stdin, writes JSON, pipes into any script or CI pipeline.
Implement IEmbeddingGenerator locally in .NET. Plug Kjarni into Microsoft.Extensions.AI and Semantic Kernel with one NuGet package that has zero dependencies. Runs on CPU, in your process, offline.
Run Llama, Mistral, Qwen or Phi locally from C#. One NuGet package, loaded in-process and answering offline. Streaming, multi-turn conversations and sampling control in .NET.
Improve search relevance in .NET with cross-encoder reranking. Rescore your existing search results from Elasticsearch, SQL LIKE, or vector search, locally in C# and in your own process.
Build a retrieval-augmented generation pipeline in C#: index documents, search them, rerank the hits, and answer with a local LLM. One NuGet package, start to finish, with every stage running on your own machine.
Implement IChatClient with the model inside your .NET process. No daemon, no localhost port. Streaming, multi-turn and Semantic Kernel.
all-MiniLM-L6-v2 truncates at 256 tokens, about 900 characters, and drops the rest without a warning. Why the configs say 128, 256 and 512, what chunk size to use, and why raising max_seq_length to 512 makes retrieval worse.
Run embeddings, classification and reranking from C++ with no libtorch and no ONNX Runtime: one header and one shared library that links only against libc. Works with C++11, C++14, C++17 and C++23, and takes four commands from nothing to output.
Index a directory, then search it by keyword, by meaning, or both, from Python. One pip install with no dependencies, running on your own machine, with a worked example of where hybrid search goes wrong and what fixes it.
Load Llama, Mistral, Qwen or Phi inside a C++ program and generate text in-process. One header and one shared library, blocking replies and token streaming, with the model running on your own machine.
Which Kjarni file you need on Windows and where it comes from: Kjarni.dll and kjarni_ffi.dll for C# and Unity, kjarni_ffi.lib and kjarni.h for C++, and kjarni.exe for the command line. Nothing else to install from 0.2.4.
Run transformer models inside a Unity game with no server and no API key: match free-text player commands by meaning, moderate chat with toxic-bert, and give an NPC a local LLM. Two DLLs, .NET Standard 2.1, Mono and IL2CPP.
Add keyword, semantic and hybrid search with cross-encoder reranking to a Qt 6 / QML application, running locally in C++ with no Python, no ONNX Runtime and no server. A complete working example with real results.