Why I Built a Native ML Inference Engine in Rust
Kjarni is a single native library that runs transformer models in your own process, from Rust, C#, Go, C++ or the command line. One install, on CPU, offline after the first run.
Read more6 posts
Kjarni is a single native library that runs transformer models in your own process, from Rust, C#, Go, C++ or the command line. One install, on CPU, offline after the first run.
Read moreRun Llama, Mistral, Qwen or Phi locally from C#. One NuGet package, loaded in-process and answering offline. Streaming, multi-turn conversations and sampling control in .NET.
Read moreImplement IChatClient with the model inside your .NET process. No daemon, no localhost port. Streaming, multi-turn and Semantic Kernel.
Read moreRun embeddings, classification and reranking from C++ with no libtorch and no ONNX Runtime: one header and one shared library that links only against libc. Works with C++11, C++14, C++17 and C++23, and takes four commands from nothing to output.
Read moreLoad Llama, Mistral, Qwen or Phi inside a C++ program and generate text in-process. One header and one shared library, blocking replies and token streaming, with the model running on your own machine.
Read moreWhich Kjarni file you need on Windows and where it comes from: Kjarni.dll and kjarni_ffi.dll for C# and Unity, kjarni_ffi.lib and kjarni.h for C++, and kjarni.exe for the command line. Nothing else to install from 0.2.4.
Read more