Documentation · Documentation
Hardware
CPU and GPU support, what has been verified on the GPU, and how Kjarni runs in the browser.
Everything runs on CPU by default, using all physical cores within a call, with SIMD kernels for x86-64 (AVX2) and ARM (NEON), and int4/int8 quantized weights for chat models.
GPU support goes through WebGPU (the wgpu library), so it uses Vulkan, DirectX 12 or
Metal and works on AMD, Intel, Apple and NVIDIA hardware without a CUDA toolkit. Pass
gpu: true (or --gpu, or device: "gpu" depending on the binding).
| Model type | GPU status |
|---|---|
| Encoders: embeddings, classification, reranking | Output matches PyTorch, within 6e-6 |
| Chat and generation | Full-precision Qwen matches PyTorch; quantized files can produce different text on the GPU than on the CPU |
| Speech to text: Whisper, from 0.2.8 | Same transcript as the CPU on the whisper-small test clips |
| Summarization and translation | Less tested; prefer CPU |
The GPU is not the fastest option on an NVIDIA card: CUDA engines are well ahead on the same hardware, as the benchmarks show. What it adds is that one build runs on any vendor's GPU with nothing else to install.
In the browser, kjarni-wasm runs on CPU with WebAssembly SIMD, in a Web Worker so the
page does not freeze. A WebGPU build of the engine exists in the repository but is not in
the npm package yet.
Kjarni runs the same engine on every platform: