Sentence Embeddings in Python without PyTorch: all-MiniLM-L6-v2 in 23 MB

pip install sentence-transformers brings PyTorch with it. On a CPU-only server that is 1.4 GB of packages to compute 384 numbers per sentence. It is fine on a workstation and a problem in a Docker image, an AWS Lambda function or a CI job that installs it every run.

The same model runs without any of it:

pip install kjarni

from kjarni import Embedder

model = Embedder("minilm-l6-v2")

docs = [
    "How do I reset my password?",
    "Where can I track my order?",
    "What is your refund policy?",
    "How do I update my billing address?",
    "Do you ship internationally?",
]
query = "I need to change my login credentials"

for doc in docs:
    print(f"{model.similarity(query, doc):7.4f}  {doc}")

 0.5981  How do I reset my password?
 0.0767  Where can I track my order?
-0.0027  What is your refund policy?
 0.4067  How do I update my billing address?
-0.0451  Do you ship internationally?

Those are the scores sentence-transformers gives for the same sentences, to all four decimals. The first run downloads the model (90 MB) from Hugging Face and caches it.

The same model, side by side

The sentence-transformers version, for comparison:

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2", device="cpu")
q = model.encode("I need to change my login credentials")
for doc in docs:
    print(f"{float(util.cos_sim(q, model.encode(doc))):7.4f}  {doc}")

It prints the same five numbers. Measured in two fresh virtual environments on Python 3.13, Linux x86-64, with PyTorch from its CPU-only index:

An empty virtual environment is 12 MB, so Kjarni adds 23 MB. The PyTorch figure is the CPU-only build; the default PyTorch wheel on Linux adds the CUDA libraries on top.

The last row goes the other way: PyTorch encodes a large batch about 20% faster here. If you embed millions of sentences in one long-running job, that matters more than start-up time. For a web service, a serverless function, a CLI or a CI step, where the process starts often and encodes little each time, the 3.4 seconds of import and load usually matter more.

Batches and numpy

vectors = model.encode_batch(["first sentence", "second sentence"])
print(vectors.shape)   # (2, 384)

encode_batch returns a numpy array when numpy is installed, and a list of lists when it is not. Kjarni does not depend on numpy, so it is never installed for you. Vectors are L2-normalized by default, so a dot product is the cosine similarity; pass Embedder("minilm-l6-v2", normalize=False) for the raw vectors.

In a Docker image

FROM python:3.12-slim
RUN pip install --no-cache-dir kjarni
# Download the model at build time, so containers start offline.
RUN python -c "from kjarni import Embedder; Embedder('minilm-l6-v2')"
COPY app.py .
CMD ["python", "app.py"]

With an app.py of:

from kjarni import Embedder

model = Embedder("minilm-l6-v2")
score = model.similarity("I need to change my login credentials", "How do I reset my password?")
print(f"{score:.4f}")

docker build -t embed-test .
docker run --rm --network none embed-test

0.5981

--network none shows the container needs nothing from the internet once built. The image is 243 MB, base image and model included.

The wheel holds a native library built for glibc on Linux x86-64 and arm64, so the -slim images work. Alpine uses musl and does not.

What else it runs

The same Embedder loads other sentence-transformers models by name: mpnet-base-v2 (768 dimensions, better quality, slower), multilingual-e5-small and bge-m3 for other languages, and nomic-embed-text for longer inputs. They download on first use like MiniLM. Models load by their name in Kjarni's list, not by an arbitrary Hugging Face repository id.

The package also has Classifier, Reranker, Indexer and Searcher for search, Chat for local LLMs and Transcriber for Whisper, all without PyTorch.

What it does not do

• No fine-tuning. Kjarni runs models; it does not train them.
• Inputs are cut at 256 tokens, the same as sentence-transformers. About 900 characters of English. See all-MiniLM-L6-v2 Max Sequence Length for what that does to long documents and how to chunk.
• No musl. Alpine needs the glibc image or a different base.

FAQ

Can I use all-MiniLM-L6-v2 without PyTorch?

Yes. pip install kjarni runs it from a native library, with no PyTorch, no transformers and no ONNX export, and gives the same similarity scores as sentence-transformers to four decimals.

How much smaller is it than sentence-transformers?

23 MB installed against 1.4 GB for sentence-transformers with CPU-only PyTorch, measured in fresh virtual environments. The CUDA build of PyTorch is larger still.

Is it faster?

It starts much faster: 0.07 seconds to import and load, against about 3.4 seconds. On a large batch PyTorch is about 20% faster on the same machine.

Does it need a GPU?

No. It runs on the CPU. Embedder("minilm-l6-v2", device="gpu") uses a GPU through Vulkan, Metal or DirectX 12 when one is present.

Getting it

Install: pip install kjarni
PyPI:    https://pypi.org/project/kjarni/
GitHub:  https://github.com/olafurjohannsson/kjarni
Example: https://github.com/kjarni-ai/kjarni-examples/tree/main/python-semantic-search

Related

• Local Semantic Search in Python: index a folder and search it by keyword, meaning or both
• all-MiniLM-L6-v2 Max Sequence Length: 256 tokens, not 512
• Benchmarking against PyTorch: how the outputs are checked