C++ API

Kjarni for C++23.

A header-only wrapper over kjarni.h. Nothing here allocates on its own account: every handle is a std::unique_ptr with the matching C deleter, and every array the C API hands back is owned by a move-only RAII type. Leaking or double-freeing takes deliberate effort.

Errors are values. Fallible calls return std::expected<T, kjarni::Error> rather than throwing, so a caller decides whether a missing model is exceptional. Nothing in this header throws except std::bad_alloc.

#include "kjarni.hpp"
auto embedder = kjarni::Embedder::create({.model = "minilm-l6-v2"});
if (!embedder) return std::println("{}", embedder.error().message()), 1;
auto vec = embedder->encode("semantic search, locally");
std::println("{} dimensions", vec->size());

Requires C++23 for std::expected. For C++20, swap std::expected for a std::variant or exceptions; everything else here is C++20.

Types

Error

A failed call, carrying the library's own diagnostic text.

The message is captured at construction rather than read lazily: the C API keeps one global "last error", so reading it later would report whatever failed most recently instead of what failed here.

Error

explicit Error(KjarniErrorCode code) : code_(code), message_(capture()) {} KjarniErrorCode code() const noexcept

code

KjarniErrorCode code() const noexcept

message

const std::string& message() const noexcept

Result

The result of any fallible Kjarni call.

using Result = std::expected<T, Error>;

Embedding

A single embedding. Borrowed as a std::span, so no copy is made.

Embedding

explicit Embedding(KjarniFloatArray arr) noexcept : arr_(arr) {} std::span<const float> values() const noexcept

values

std::span<const float> values() const noexcept

size

std::size_t size() const noexcept

data

const float* data() const noexcept

begin

auto begin() const noexcept

end

auto end() const noexcept

to_vector

std::vector<float> to_vector() const

Copies into an owning vector, for when the embedding must outlive this object.

Label

One predicted label.

std::string name;
float score{};

Ranked

One reranked document: an index into the input, and its score.

std::size_t index{};
float score{};

Hit

One search hit.

float score{};
std::size_t document_id{};
std::string text;
std::string metadata_json;

EmbedderOptions

Turns text into vectors. Designated-initialiser options: Embedder::create({.model = "mpnet-base-v2"}).

std::string model = "minilm-l6-v2";
std::string cache_dir;      ///< empty = default location
bool gpu = false;
bool normalize = true;
bool quiet = true;

Embedder

create

static Result<Embedder> create(const EmbedderOptions& opts = {})

encode

Result<Embedding> encode(std::string_view text) const

encode

Result<std::vector<std::vector<float>>> encode( std::span<const std::string> texts) const

Encodes a batch in one call, which is markedly faster than looping.

dimensions

std::size_t dimensions() const noexcept

ClassifierOptions

Sentiment, emotion and toxicity classification.

std::string model = "distilbert-sentiment";
std::string cache_dir;
bool gpu = false;
bool quiet = true;

Classifier

create

static Result<Classifier> create(const ClassifierOptions& opts = {})

classify

Result<std::vector<Label>> classify(std::string_view text) const

Every label with its score, highest first.

classify

Result<std::vector<std::vector<Label>>> classify(std::span<const std::string> texts) const

Every label with its score for each text, in order. One call for many texts is much faster than calling classify for each.

top

Result<std::optional<Label>> top(std::string_view text) const

The highest-scoring label, or std::nullopt if the model returned nothing.

num_labels

std::size_t num_labels() const noexcept

RerankerOptions

Cross-encoder reranking: rescores candidates you already have.

std::string model = "minilm-l6-v2-cross-encoder";
std::string cache_dir;
bool gpu = false;
bool quiet = true;

Reranker

create

static Result<Reranker> create(const RerankerOptions& opts = {})

score

Result<float> score(std::string_view query, std::string_view document) const

Scores one query-document pair. Raw logits: negative values are normal, and only the ordering is meaningful.

rerank

Result<std::vector<Ranked>> rerank(std::string_view query, std::span<const std::string> documents) const

Reorders documents, best first. Ranked::index points back into the input, so the caller keeps its own IDs and metadata.

rerank

Result<std::vector<Ranked>> rerank(std::string_view query, std::span<const std::string> documents, std::size_t k) const

Reranks and keeps only the best k.

Generation

Sampling controls. Defaults come from the model.

std::optional<float> temperature;
std::optional<int> top_k;
std::optional<float> top_p;
std::optional<int> max_new_tokens;
/// Deterministic output: same prompt, same answer.
///
/// Fields are assigned rather than designated-initialised so that a partial
/// initialiser list does not warn under `-Wextra` in consumers' builds.
[[nodiscard]] static Generation greedy(int max_tokens = 512) {
    Generation g;
    g.temperature = 0.f;
    g.max_new_tokens = max_tokens;
    return g;
}
[[nodiscard]] KjarniGenerationConfig to_c() const noexcept {
    KjarniGenerationConfig cfg = kjarni_generation_config_default();
    if (temperature) {
        cfg.temperature = *temperature;
        cfg.do_sample = *temperature > 0.f ? 1 : 0;
    }
    if (top_k) cfg.top_k = *top_k;
    if (top_p) cfg.top_p = *top_p;
    if (max_new_tokens) cfg.max_new_tokens = *max_new_tokens;
    return cfg;
}

ChatOptions

A local language model.

Generation is compute-bound and the underlying model is not re-entrant, so one instance serves one request at a time. Share it behind your own queue rather than creating several.

std::string model = "llama3.2-3b-instruct";
std::string system_prompt;
bool gpu = false;
bool quiet = true;
/// Load the 4-bit GGUF weights instead of full precision: a fraction of the
/// memory and several times faster to generate on a CPU.
bool quantized = false;

Chat

create

static Result<Chat> create(const ChatOptions& opts = {})

send

Result<std::string> send(std::string_view message) const

Generates a full response before returning.

send

Result<std::string> send(std::string_view message, const Generation& gen) const

stream

template <typename F> std::expected<void, Error> stream(std::string_view message, F&& on_token) const

Streams tokens as they are produced.

on_token is any callable taking std::string_view. Return false from it to stop generation early rather than running to completion and discarding the result.

stream

template <typename F> std::expected<void, Error> stream(std::string_view message, const Generation& gen, F&& on_token) const

context_size

std::size_t context_size() const noexcept

The model's context window, in tokens.

IndexStats

What an indexing run did.

std::size_t documents_indexed{};
std::size_t chunks_created{};
std::size_t dimension{};
std::uint64_t size_bytes{};
std::size_t files_processed{};
std::size_t files_skipped{};
std::uint64_t elapsed_ms{};

IndexerOptions

std::string model = "minilm-l6-v2";
std::string cache_dir;
/// Characters per chunk; 0 keeps the library's default.
std::size_t chunk_size = 0;
/// Characters shared by neighbouring chunks; 0 keeps the library's default.
std::size_t chunk_overlap = 0;
/// File extensions to index, e.g. {"md", "txt"}; empty keeps the defaults.
std::vector<std::string> extensions;
/// Names or `*` patterns to skip, e.g. {"node_modules", "*.min.js"}.
std::vector<std::string> exclude;
bool recursive = true;
bool include_hidden = false;
bool gpu = false;
bool quiet = true;

Indexer

Builds a search index on disk from files and folders.

create

static Result<Indexer> create(const IndexerOptions& opts = {})

create_index

Result<IndexStats> create_index(std::string_view index_path, std::span<const std::string> inputs, bool force = false) const

Indexes inputs (files and folders) into a new index at index_path. force replaces an index already there.

add

Result<std::size_t> add(std::string_view index_path, std::span<const std::string> inputs) const

Adds inputs to an existing index; returns how many chunks were added.

SearchMode

enum class SearchMode { keyword = KJARNI_SEARCH_MODE_KEYWORD, semantic = KJARNI_SEARCH_MODE_SEMANTIC, hybrid = KJARNI_SEARCH_MODE_HYBRID };

SearcherOptions

/// Must be the model the index was built with.
std::string model = "minilm-l6-v2";
/// A cross-encoder for reranking, e.g. "minilm-l6-v2-cross-encoder"; empty for none.
std::string rerank_model;
std::string cache_dir;
SearchMode mode = SearchMode::hybrid;
std::size_t top_k = 10;
bool gpu = false;
bool quiet = true;

SearchOptions

Per-query overrides; anything left unset uses the searcher's defaults.

std::optional<SearchMode> mode;
std::size_t top_k = 0;
std::optional<bool> rerank;
/// Drop results scoring below this.
float threshold = 0.f;
/// Only files whose path matches this `*` pattern.
std::string source_pattern;
std::string filter_key;
std::string filter_value;

Searcher

Searches an index built by Indexer, by keyword, meaning or both.

create

static Result<Searcher> create(const SearcherOptions& opts = {})

search

Result<std::vector<Hit>> search(std::string_view index_path, std::string_view query) const

search

Result<std::vector<Hit>> search(std::string_view index_path, std::string_view query, const SearchOptions& options) const

has_reranker

bool has_reranker() const noexcept

Segment

One timed piece of a transcript, in seconds from the start.

float start{};
float end{};
std::string text;

Transcript

std::string text;
/// ISO code of the language transcribed: the one given, or the one detected.
std::string language;
float duration_secs{};
std::vector<Segment> segments;

TranscriberOptions

std::string model = "whisper-small";
/// ISO code such as "en" or "is"; empty detects it from the audio.
std::string language;
/// Translate the speech into English instead of transcribing it.
bool translate = false;
bool gpu = false;
bool quiet = true;

Transcriber

Speech to text with Whisper.

create

static Result<Transcriber> create(const TranscriberOptions& opts = {})

transcribe_file

Result<Transcript> transcribe_file(std::string_view path) const

Transcribes a wav, mp3, flac or ogg file.

transcribe

Result<Transcript> transcribe(std::span<const float> samples, std::uint32_t sample_rate) const

Transcribes mono samples in -1..1 at any sample rate.

SummarizerOptions

std::string model = "distilbart-cnn";
std::string cache_dir;
/// Summary length bounds in tokens; 0 keeps the model's defaults.
std::size_t min_length = 0;
std::size_t max_length = 0;
bool gpu = false;
bool quiet = true;

Summarizer

Summarizes article-length text.

create

static Result<Summarizer> create(const SummarizerOptions& opts = {})

summarize

Result<std::string> summarize(std::string_view text) const

TranslatorOptions

std::string model = "flan-t5-base";
std::string cache_dir;
/// Default languages for `translate(text)`, e.g. "en" and "de".
std::string from;
std::string to;
bool gpu = false;
bool quiet = true;

Translator

Translates text between languages.

create

static Result<Translator> create(const TranslatorOptions& opts = {})

translate

Result<std::string> translate(std::string_view text) const

Translates between the default languages given at creation.

translate

Result<std::string> translate(std::string_view text, std::string_view from, std::string_view to) const

ImageHit

std::string path;
/// Cosine similarity; small even for good matches, so only the order matters.
float score{};

ScanReport

std::size_t added{};
/// Files that could not be decoded.
std::size_t skipped{};

ImageIndexOptions

/// A CLIP model directory; empty uses clip-vit-base-32, downloaded on first use.
std::string model_dir;

ImageIndex

Photos searched by description, with CLIP. Not safe to share across threads while adding.

create

static Result<ImageIndex> create(const ImageIndexOptions& opts = {})

add

std::expected<void, Error> add(std::string_view path)

add_directory

Result<ScanReport> add_directory(std::string_view dir)

Adds every JPEG and PNG under dir.

search

Result<std::vector<ImageHit>> search(std::string_view query, std::size_t top_k = 10) const

save

std::expected<void, Error> save(std::string_view path) const

Writes the vectors (not the images) to path.

load

Result<std::size_t> load(std::string_view path)

Restores vectors saved by save; returns how many.

size

std::size_t size() const noexcept

Functions

cosine

float cosine(std::span<const float> a, std::span<const float> b) noexcept

Cosine similarity between two embeddings. Vectors of unequal length compare over the shorter prefix, which is what the C function does; mismatched dimensions almost always mean two different models.

cosine

float cosine(const Embedding& a, const Embedding& b) noexcept

version

std::string_view version() noexcept

Library version, e.g. "0.1.0".