Why I Built a Native ML Inference Engine in Rust
Kjarni is a single native library that runs transformer models in your own process, from Rust, C#, Go, C++ or the command line. One install, on CPU, offline after the first run.
Read more6 posts
Kjarni is a single native library that runs transformer models in your own process, from Rust, C#, Go, C++ or the command line. One install, on CPU, offline after the first run.
Read moreAdd semantic search to your C# app in 5 lines. Match text by meaning, not keywords. One NuGet package with zero dependencies, running on CPU in your own process.
Read moreRun sentiment analysis, generate embeddings, detect toxicity, and search documents from your terminal. One binary, reads stdin, writes JSON, pipes into any script or CI pipeline.
Read moreImplement IEmbeddingGenerator locally in .NET. Plug Kjarni into Microsoft.Extensions.AI and Semantic Kernel with one NuGet package that has zero dependencies. Runs on CPU, in your process, offline.
Read moreThree config files disagree about how long an all-MiniLM-L6-v2 input can be, and the answer sentence-transformers uses is 256 tokens. At a 1000-character chunk size, 87% of chunks are longer than that, and the excess is discarded silently.
Read moreFive embedding models, both entry points, batches from 1 to 64, measured against PyTorch on the same machine. Encoding is 1.0x to 1.9x behind, decode is 1.6x ahead, and the reason is one library nobody ships in a self-contained binary.
Read more