kjarni-go

Package kjarni runs transformer models inside your Go program: text classification, embeddings, semantic search, reranking, chat, speech to text, summarization, translation and image search, with the same output as PyTorch and no Python or server. Kjarni is a transformer inference library; this package calls its native library. Models download automatically on first use and are cached on disk, so everything after that works offline.

The same engine ships for other runtimes: Kjarni on NuGet for C#, kjarni-wasm on npm for the browser, and a command line binary. Documentation and guides live at https://kjarni.ai and the source is at https://github.com/olafurjohannsson/kjarni

Getting started

import "github.com/olafurjohannsson/kjarni-go"

emb, err := kjarni.NewEmbedder("minilm-l6-v2")
if err != nil {
	log.Fatal(err)
}
defer emb.Close()

vec, err := emb.Encode("hello world")

Beyond text embeddings

tr, _ := kjarni.NewTranscriber("", kjarni.WithLanguage("en"))
transcript, _ := tr.TranscribeFile("meeting.wav")

sum, _ := kjarni.NewSummarizer("")
summary, _ := sum.Summarize(article)

tl, _ := kjarni.NewTranslator("", kjarni.WithLanguages("en", "de"))
german, _ := tl.Translate("The house is small.")

photos, _ := kjarni.NewImageIndex("")
photos.AddDirectory("Pictures")
hits, _ := photos.Search("a red car", 5)

Functions

CosineSimilarity

func CosineSimilarity(a, b []float32) float32

CosineSimilarity computes cosine similarity between two vectors in Go.

Types

Chat

type Chat struct {
	// contains filtered or unexported fields
}

Chat runs a local language model in-process. It is safe for concurrent use; calls are serialized. Close releases the model.

NewChat

func NewChat(model string, opts ...interface{}) (*Chat, error)

NewChat loads a chat model by registry name, for example "llama3.2-3b-instruct", downloading it on first use.

opts takes ChatOption values (WithSystemPrompt, WithChatMode, WithQuantized) and, of the shared options, WithQuiet and WithDevice.

Chat.Close

func (c *Chat) Close() error

releases the Chat resources

Chat.ContextSize

func (c *Chat) ContextSize() int

returns the model's context window size

Chat.Conversation

func (c *Chat) Conversation() (*ChatConversation, error)

create a stateful conversation with history management

Chat.ModelName

func (c *Chat) ModelName() string

returns model name

Chat.Send

func (c *Chat) Send(message string) (string, error)

sends a message and returns the full response, blocking

Chat.SendWithConfig

func (c *Chat) SendWithConfig(message string, config *GenerationConfig) (string, error)

sends a message with custom generation config

Chat.SendWithHistory

func (c *Chat) SendWithHistory(turns []ChatTurn, message string, config *GenerationConfig) (string, error)

SendWithHistory replies to message given prior turns, oldest first, without keeping any state. It suits callers that own the transcript and replay it every turn; ChatConversation is simpler otherwise. A RoleSystem turn replaces the system prompt given to NewChat. config may be nil for the defaults.

Chat.Stream

func (c *Chat) Stream(message string, onToken func(token string) bool) error

Stream sends a message and calls onToken for each generated token. Return false from onToken to stop generation.

Chat.StreamWithConfig

func (c *Chat) StreamWithConfig(message string, config *GenerationConfig, onToken func(token string) bool) error

streams with custom generation config.

Chat.StreamWithHistory

func (c *Chat) StreamWithHistory(turns []ChatTurn, message string, config *GenerationConfig, onToken func(token string) bool) error

StreamWithHistory is SendWithHistory, streamed through onToken. Return false from onToken to stop.

ChatConversation

type ChatConversation struct {
	// contains filtered or unexported fields
}

ChatConversation maintains chat history parent Chat must outlive this object

ChatConversation.Clear

func (cc *ChatConversation) Clear(keepSystem bool)

clear the conversation history

ChatConversation.Close

func (cc *ChatConversation) Close() error

release the conversation resources

ChatConversation.Length

func (cc *ChatConversation) Length() int

returns the number of messages in history

ChatConversation.Send

func (cc *ChatConversation) Send(message string) (string, error)

Send sends a message and returns a response Both user message and response are added to history

ChatConversation.SendWithConfig

func (cc *ChatConversation) SendWithConfig(message string, config *GenerationConfig) (string, error)

send with custom generation config.

ChatConversation.Stream

func (cc *ChatConversation) Stream(message string, onToken func(token string) bool) error

streams a response. Message and full response are added to history

ChatMode

type ChatMode int32

ChatMode picks a chat's sampling defaults.

const (
	// ChatDefault samples at temperature 0.7, up to 512 new tokens.
	ChatDefault	ChatMode	= 0
	// ChatCreative samples at temperature 0.9, up to 1024 new tokens.
	ChatCreative	ChatMode	= 1
	// ChatReasoning samples at temperature 0.3, up to 2048 new tokens.
	ChatReasoning	ChatMode	= 2
)

ChatOption

type ChatOption func(*chatOptions)

ChatOption configures NewChat.

WithChatMode

func WithChatMode(mode ChatMode) ChatOption

WithChatMode picks the sampling defaults; see ChatMode.

WithQuantized

func WithQuantized(quantized bool) ChatOption

WithQuantized loads the model's 4-bit GGUF weights instead of full precision: a fraction of the memory and several times faster to generate on a CPU.

WithSystemPrompt

func WithSystemPrompt(prompt string) ChatOption

WithSystemPrompt sets the instructions the model sees before every message.

ChatRole

type ChatRole int32

ChatRole is who wrote a turn of a transcript.

const (
	RoleSystem	ChatRole	= 0
	RoleUser	ChatRole	= 1
	RoleAssistant	ChatRole	= 2
)

ChatTurn

type ChatTurn struct {
	Role	ChatRole
	Content	string
}

ChatTurn is one message of a transcript the caller keeps.

Classifier

type Classifier struct {
	// contains filtered or unexported fields
}

Classifier runs text classification using a pre-trained model.

NewClassifier

func NewClassifier(model string, opts ...Option) (*Classifier, error)

NewClassifier creates a classifier for the given model. Available models: distilbert-sentiment, roberta-sentiment, bert-sentiment-multilingual, distilroberta-emotion, roberta-emotions, toxic-bert. Models download automatically on first use and are cached locally.

Classifier.Classify

func (c *Classifier) Classify(text string) (*ClassifyResult, error)

Classify runs the model on the given text and returns scored labels.

Classifier.ClassifyBatch

func (c *Classifier) ClassifyBatch(texts []string) ([]*ClassifyResult, error)

ClassifyBatch classifies several texts in one call, which is much faster than one at a time. It returns one result per text, in order.

Classifier.Close

func (c *Classifier) Close() error

Close releases the classifier resources. Safe to call multiple times.

Classifier.NumLabels

func (c *Classifier) NumLabels() int

NumLabels returns the number of labels the model supports.

ClassifyResult

type ClassifyResult struct {
	Label		string
	Score		float32
	AllScores	[]LabelScore
}

ClassifyResult holds the output of a classification. Label and Score contain the top prediction. AllScores contains scores for every label.

ClassifyResult.String

func (r *ClassifyResult) String() string

String returns the result as "label (score%)".

ClassifyResult.ToJSON

func (r *ClassifyResult) ToJSON() string

ToJSON returns the result as a JSON string.

Embedder

type Embedder struct {
	// contains filtered or unexported fields
}

Embedder encodes text into vector embeddings for similarity and search.

NewEmbedder

func NewEmbedder(model string, opts ...Option) (*Embedder, error)

NewEmbedder creates an embedder for the given model. Available models: minilm-l6-v2 (384d), mpnet-base-v2 (768d), distilbert-base (768d). Models download automatically on first use and are cached locally.

Embedder.Close

func (e *Embedder) Close() error

Close releases the embedder resources. Safe to call multiple times.

Embedder.Dim

func (e *Embedder) Dim() int

Dim returns the dimensionality of the embedding model.

Embedder.Encode

func (e *Embedder) Encode(text string) ([]float32, error)

Encode returns the embedding vector for the given text.

Embedder.EncodeBatch

func (e *Embedder) EncodeBatch(texts []string) ([][]float32, error)

EncodeBatch encodes multiple texts and returns their embedding vectors.

Embedder.Similarity

func (e *Embedder) Similarity(a, b string) (float32, error)

Similarity returns the cosine similarity between two texts, computed by the engine.

ErrorCode

type ErrorCode int32

ErrorCode represents error codes returned by the kjarni engine.

const (
	ErrOk			ErrorCode	= 0
	ErrNullPointer		ErrorCode	= 1
	ErrInvalidUtf8		ErrorCode	= 2
	ErrModelNotFound	ErrorCode	= 3
	ErrLoadFailed		ErrorCode	= 4
	ErrInferenceFailed	ErrorCode	= 5
	ErrGpuUnavailable	ErrorCode	= 6
	ErrInvalidConfig	ErrorCode	= 7
	ErrCancelled		ErrorCode	= 8
	ErrTimeout		ErrorCode	= 9
	ErrStreamEnded		ErrorCode	= 10
	// ErrPanic reports a panic in the native engine caught at the ABI boundary. The
	// operation failed and its outputs are undefined; the handle stays valid to Close.
	ErrPanic	ErrorCode	= 11
	ErrUnknown	ErrorCode	= 255
)

GenerationConfig

type GenerationConfig struct {
	// Temperature: higher is more random.
	Temperature	float32
	// TopK samples only from the k most likely tokens.
	TopK	int32
	// TopP samples from the smallest set of tokens whose probability sums to p.
	TopP	float32
	// MinP drops tokens less likely than this fraction of the most likely one.
	MinP	float32
	// RepetitionPenalty above 1 discourages repeating earlier tokens; 1 disables it.
	RepetitionPenalty	float32
	// MaxNewTokens caps the length of the reply.
	MaxNewTokens	int32
	// DoSample is 1 to sample, 0 to decode greedily, -1 for the default.
	DoSample	int32
}

GenerationConfig controls how a reply is generated. A negative value keeps the model's default for that field.

CreativeGenerationConfig

func CreativeGenerationConfig() GenerationConfig

CreativeGenerationConfig samples at temperature 0.9 with top-k 50 and top-p 0.95.

DefaultGenerationConfig

func DefaultGenerationConfig() GenerationConfig

DefaultGenerationConfig keeps every model default.

GreedyGenerationConfig

func GreedyGenerationConfig() GenerationConfig

GreedyGenerationConfig always picks the most likely token, so the same prompt gives the same reply every time.

ImageHit

type ImageHit struct {
	Path	string
	// Score is a cosine similarity, small even for good matches, so only the
	// order is meaningful.
	Score	float32
}

ImageHit is an image matching a search.

ImageIndex

type ImageIndex struct {
	// contains filtered or unexported fields
}

ImageIndex searches photos by description, with CLIP.

NewImageIndex

func NewImageIndex(modelDir string) (*ImageIndex, error)

NewImageIndex loads a CLIP model from modelDir, or clip-vit-base-32 for "", downloaded on first use.

ImageIndex.Add

func (x *ImageIndex) Add(path string) error

Add indexes one JPEG or PNG.

ImageIndex.AddDirectory

func (x *ImageIndex) AddDirectory(dir string) (ScanReport, error)

AddDirectory indexes every JPEG and PNG under dir.

ImageIndex.Close

func (x *ImageIndex) Close() error

Close releases the model. Safe to call more than once.

ImageIndex.Len

func (x *ImageIndex) Len() int

Len reports how many images are indexed.

ImageIndex.Load

func (x *ImageIndex) Load(path string) (int, error)

Load restores vectors written by Save and returns how many.

ImageIndex.Save

func (x *ImageIndex) Save(path string) error

Save writes the vectors (not the images) to path.

ImageIndex.Search

func (x *ImageIndex) Search(query string, topK int) ([]ImageHit, error)

Search returns the topK images best matching query.

IndexStats

type IndexStats struct {
	DocumentsIndexed	int
	ChunksCreated		int
	Dimension		int
	SizeBytes		uint64
	FilesProcessed		int
	FilesSkipped		int
	ElapsedMs		uint64
}

IndexStats holds statistics from an indexing operation.

Indexer

type Indexer struct {
	// contains filtered or unexported fields
}

Indexer creates search indexes from files in a directory.

NewIndexer

func NewIndexer(model string, opts ...Option) (*Indexer, error)

NewIndexer creates an indexer using the given embedding model. The model is used to generate vectors for each text chunk during indexing.

Indexer.Add

func (idx *Indexer) Add(indexPath string, inputs []string) (int, error)

Add indexes more files and directories into the existing index at indexPath, returning how many chunks were added.

Indexer.Close

func (idx *Indexer) Close() error

Close releases the indexer resources. Safe to call multiple times.

Indexer.Create

func (idx *Indexer) Create(indexPath string, inputs []string) (*IndexStats, error)

Create builds a new search index at indexPath from the given files and directories. Files are chunked, embedded, and stored for later retrieval with a Searcher. It fails if an index already exists there; see Recreate.

Indexer.Recreate

func (idx *Indexer) Recreate(indexPath string, inputs []string) (*IndexStats, error)

Recreate is Create, replacing any index already at indexPath.

KjarniError

type KjarniError struct {
	Code	ErrorCode
	Message	string
}

KjarniError is an error returned by the kjarni engine.

KjarniError.Error

func (e *KjarniError) Error() string

Error returns the engine's message and error code.

LabelScore

type LabelScore struct {
	Label	string
	Score	float32
}

LabelScore is a single label with its confidence score.

Option

type Option func(*options)

Option configures a classifier, embedder, or other kjarni component.

WithCacheDir

func WithCacheDir(dir string) Option

WithCacheDir sets where models are downloaded to and loaded from.

WithChunking

func WithChunking(size, overlap int) Option

WithChunking sets an Indexer's chunk size and the overlap between chunks, in characters.

WithDevice

func WithDevice(device string) Option

WithDevice sets the compute device. Supported values: "cpu", "gpu".

WithExclude

func WithExclude(patterns ...string) Option

WithExclude makes an Indexer skip names or `*` patterns, e.g. "node_modules", "*.min.js".

WithExtensions

func WithExtensions(extensions ...string) Option

WithExtensions limits an Indexer to files with these extensions, e.g. "md", "txt".

WithHidden

func WithHidden(include bool) Option

WithHidden makes an Indexer include hidden files and directories.

WithLanguage

func WithLanguage(language string) Option

WithLanguage sets the spoken language for a Transcriber as an ISO code such as "en" or "is". Without it the language is detected from the audio.

WithLanguages

func WithLanguages(from, to string) Option

WithLanguages sets a Translator's default source and target languages.

WithQuiet

func WithQuiet(quiet bool) Option

WithQuiet suppresses log output during model loading and inference.

WithRecursive

func WithRecursive(recursive bool) Option

WithRecursive sets whether an Indexer descends into subdirectories (the default).

WithSummaryLength

func WithSummaryLength(min, max int) Option

WithSummaryLength bounds a Summarizer's output in tokens; 0 keeps the model's default.

WithTranslate

func WithTranslate(translate bool) Option

WithTranslate makes a Transcriber translate the speech into English.

RerankResult

type RerankResult struct {
	Index		int
	Score		float32
	Document	string
}

RerankResult holds a single reranked document with its relevance score.

Reranker

type Reranker struct {
	// contains filtered or unexported fields
}

Reranker scores query-document relevance using a cross-encoder model.

NewReranker

func NewReranker(opts ...Option) (*Reranker, error)

NewReranker creates a reranker using the default cross-encoder model. The model downloads automatically on first use and is cached locally.

Reranker.Close

func (r *Reranker) Close() error

Close releases the reranker resources. Safe to call multiple times.

Reranker.Rerank

func (r *Reranker) Rerank(query string, documents []string) ([]RerankResult, error)

Rerank scores all documents and returns them sorted by relevance to the query.

Reranker.RerankTopK

func (r *Reranker) RerankTopK(query string, documents []string, k int) ([]RerankResult, error)

RerankTopK scores all documents and returns the top k sorted by relevance.

Reranker.Score

func (r *Reranker) Score(query, document string) (float32, error)

Score returns the relevance score for a single query-document pair.

ScanReport

type ScanReport struct {
	Added	int
	// Skipped counts files that could not be decoded.
	Skipped	int
}

ScanReport says what scanning a directory did.

SearchMode

type SearchMode int

SearchMode determines the search strategy.

const (
	// Keyword uses BM25 term matching.
	Keyword	SearchMode	= 0
	// Semantic uses vector similarity.
	Semantic	SearchMode	= 1
	// Hybrid combines BM25 and vector similarity.
	Hybrid	SearchMode	= 2
)

SearchOptions

type SearchOptions struct {
	// TopK is the most results to return; 0 means 10.
	TopK	int
	// NoRerank skips the reranker for this query when the Searcher has one.
	NoRerank	bool
	// Threshold drops results scoring below it.
	Threshold	float32
	// SourcePattern keeps only chunks from files matching this `*` pattern, e.g. "*.md".
	SourcePattern	string
	// FilterKey and FilterValue keep only chunks whose metadata has that pair.
	FilterKey	string
	FilterValue	string
}

SearchOptions overrides a Searcher's defaults for one query. The zero value changes nothing.

SearchResult

type SearchResult struct {
	Score		float32
	Text		string
	DocumentID	int
	// Metadata stored with the chunk, such as "source", the file it came from.
	Metadata	map[string]any
}

SearchResult holds a single search result with its relevance score.

Searcher

type Searcher struct {
	// contains filtered or unexported fields
}

Searcher queries indexes created by an Indexer.

NewSearcher

func NewSearcher(model string, rerankerModel string, opts ...Option) (*Searcher, error)

NewSearcher creates a searcher using the given embedding model. Pass a non-empty rerankerModel to enable cross-encoder reranking of results. Pass an empty string to disable reranking.

Searcher.Close

func (s *Searcher) Close() error

Close releases the searcher resources. Safe to call multiple times.

Searcher.Search

func (s *Searcher) Search(indexPath string, query string, mode SearchMode) ([]SearchResult, error)

Search queries the index at indexPath and returns up to 10 results using the given mode, reranked when the Searcher has a reranker.

Searcher.SearchWithOptions

func (s *Searcher) SearchWithOptions(indexPath string, query string, mode SearchMode, opts SearchOptions) ([]SearchResult, error)

SearchWithOptions is Search with per-query overrides.

Segment

type Segment struct {
	Start	float32
	End	float32
	Text	string
}

Segment is one timed piece of a transcript, in seconds from the start.

Summarizer

type Summarizer struct {
	// contains filtered or unexported fields
}

Summarizer summarizes article-length text.

NewSummarizer

func NewSummarizer(model string, opts ...Option) (*Summarizer, error)

NewSummarizer loads a summarization model, downloading it on first use. Pass "" for distilbart-cnn. See WithSummaryLength.

Summarizer.Close

func (s *Summarizer) Close() error

Close releases the model. Safe to call more than once.

Summarizer.Summarize

func (s *Summarizer) Summarize(text string) (string, error)

Summarize summarizes text.

Transcriber

type Transcriber struct {
	// contains filtered or unexported fields
}

Transcriber turns speech into text with Whisper.

NewTranscriber

func NewTranscriber(model string, opts ...Option) (*Transcriber, error)

NewTranscriber loads a Whisper model, downloading it on first use. Pass "" for whisper-small. See WithLanguage and WithTranslate.

Transcriber.Close

func (t *Transcriber) Close() error

Close releases the model. Safe to call more than once.

Transcriber.Transcribe

func (t *Transcriber) Transcribe(samples []float32, sampleRate int) (*Transcript, error)

Transcribe transcribes mono samples in -1..1 at any sample rate.

Transcriber.TranscribeFile

func (t *Transcriber) TranscribeFile(path string) (*Transcript, error)

TranscribeFile transcribes a wav, mp3, flac or ogg file.

Transcript

type Transcript struct {
	Text	string
	// Language is the ISO code transcribed: the one given, or the one detected.
	Language	string
	DurationSecs	float32
	Segments	[]Segment
}

Transcript is the result of transcribing audio.

Translator

type Translator struct {
	// contains filtered or unexported fields
}

Translator translates text between languages.

NewTranslator

func NewTranslator(model string, opts ...Option) (*Translator, error)

NewTranslator loads a translation model, downloading it on first use. Pass "" for flan-t5-base. WithLanguages sets the languages Translate uses by default.

Translator.Close

func (t *Translator) Close() error

Close releases the model. Safe to call more than once.

Translator.Translate

func (t *Translator) Translate(text string) (string, error)

Translate translates text between the default languages given to NewTranslator.

Translator.TranslateBetween

func (t *Translator) TranslateBetween(text, from, to string) (string, error)

TranslateBetween translates text from one language to another, e.g. "en" to "de" or "english" to "german". An empty language falls back to the default; with neither, the error has code ErrInvalidConfig.