· 3 min read

Semantic Search in Go without cgo, Python or a Vector Database

A semantic search HTTP service in 60 lines of Go. The embedding model runs inside the Go process, the native library is embedded in the binary, and it builds with CGO_ENABLED=0. About 20 ms per search on a CPU.

Semantic search from Go usually means a second service: a Python process running sentence-transformers, an embeddings API, or a vector database with a model attached. For a few thousand documents, all of that can live inside the Go program instead.

This is a FAQ search service. Ask "can you send it to Canada" and it answers with "Do you ship internationally?", which shares no words with the question:

curl "localhost:8080/search?q=can+you+send+it+to+Canada"
[{"score":0.5700587,"question":"Do you ship internationally?"},{"score":0.39702776,"question":"Can I change my delivery address after ordering?"},{"score":0.3182982,"question":"How long does shipping take?"}]

Build it

You need Go 1.21 or newer, on Linux x86-64.

mkdir faqsearch && cd faqsearch
go mod init faqsearch
go get github.com/olafurjohannsson/[email protected]

Save this as main.go:

package main

import (
	"encoding/json"
	"log"
	"net/http"
	"sort"

	kjarni "github.com/olafurjohannsson/kjarni-go"
)

var faq = []string{
	"How do I reset my password?",
	"Where can I track my order?",
	"What is your refund policy?",
	"How do I update my billing address?",
	"Do you ship internationally?",
	"Can I change my delivery address after ordering?",
	"How long does shipping take?",
	"How do I cancel my subscription?",
}

type hit struct {
	Score    float32 `json:"score"`
	Question string  `json:"question"`
}

func main() {
	embedder, err := kjarni.NewEmbedder("minilm-l6-v2", kjarni.WithQuiet(true))
	if err != nil {
		log.Fatal(err)
	}
	defer embedder.Close()

	// Embed the questions once at startup; each request embeds only its query.
	vectors, err := embedder.EncodeBatch(faq)
	if err != nil {
		log.Fatal(err)
	}

	http.HandleFunc("/search", func(w http.ResponseWriter, r *http.Request) {
		query, err := embedder.Encode(r.URL.Query().Get("q"))
		if err != nil {
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}
		hits := make([]hit, len(faq))
		for i, v := range vectors {
			hits[i] = hit{kjarni.CosineSimilarity(query, v), faq[i]}
		}
		sort.Slice(hits, func(i, j int) bool { return hits[i].Score > hits[j].Score })
		w.Header().Set("Content-Type", "application/json")
		json.NewEncoder(w).Encode(hits[:3])
	})

	log.Println("listening on :8080")
	log.Fatal(http.ListenAndServe(":8080", nil))
}
CGO_ENABLED=0 go build -o faqsearch .
./faqsearch

The first start downloads the model, all-MiniLM-L6-v2 (90 MB), and caches it. Then, from another terminal:

curl "localhost:8080/search?q=I+forgot+my+login"
[{"score":0.6110735,"question":"How do I reset my password?"},{"score":0.29763743,"question":"How do I cancel my subscription?"},{"score":0.24557316,"question":"How do I update my billing address?"}]

No cgo

kjarni-go calls the engine through purego rather than cgo, and carries the native library inside the module with go:embed. At start-up it writes the library to a temporary directory and loads it. So:

  • CGO_ENABLED=0 builds work, with no C compiler on the build machine.
  • The result is one file. Copy faqsearch to another Linux x86-64 machine and run it; the only thing it fetches is the model, once.
  • The binary is large: 59 MB, most of it the embedded engine.

How fast

On an Intel i7-13700, 20 requests in a row took 0.46 s, about 23 ms each, measured with curl. Most of that is embedding the query; scoring eight vectors is nothing. The process used 167 MB of memory with the model loaded.

The loop over vectors is a brute-force scan, which stays fast for a few thousand entries. For a large collection, or text in files rather than in a slice, use kjarni.NewIndexer and kjarni.NewSearcher, which keep the index on disk and add keyword and hybrid search.

What the scores mean

The scores are cosine similarities between the query and each question. A good match here is around 0.55 to 0.7 and unrelated questions sit below 0.3, but the gap depends on your data. To answer "no match" instead of always returning three, check the top score against a threshold you pick from your own queries.

Limits

  • Linux x86-64 only, for now. The module embeds the library for Linux x86-64 and Windows x86-64, but 0.4.2 does not compile for Windows, and there is no macOS or arm64 library in the Go module yet. The C# package and the Python package cover those platforms.
  • English. all-MiniLM-L6-v2 was trained on English. kjarni.NewEmbedder("bge-m3") handles over 100 languages, with larger vectors and slower encoding.
  • One request at a time per embedder. The Embedder holds a mutex, so the handler is safe to call from many goroutines, but their encodes run one after another. For heavy concurrent traffic, keep a small pool of embedders.

FAQ

How do I do semantic search in Go?

Embed your documents and the query with a sentence embedding model, then rank the documents by cosine similarity to the query. kjarni-go runs the model inside the Go process, so there is no Python service or embeddings API to call.

Does it need cgo?

No. It uses purego to call the native library, so CGO_ENABLED=0 builds work.

Does it need a GPU or an internet connection?

No GPU. The internet is needed once, to download the model; after that it runs offline.

Getting it

Module:  go get github.com/olafurjohannsson/kjarni-go
Docs:    https://pkg.go.dev/github.com/olafurjohannsson/kjarni-go
GitHub:  https://github.com/olafurjohannsson/kjarni
Example: https://github.com/kjarni-ai/kjarni-examples/tree/main/go-semantic-search

Kjarni runs the same engine on every platform: