# Sentence Embeddings in Python without PyTorch: all-MiniLM-L6-v2 in 23 MB

> Run all-MiniLM-L6-v2 from Python without installing PyTorch or sentence-transformers. The same similarity scores to four decimals, 23 MB installed instead of 1.4 GB, and loaded in 0.07 seconds instead of 3.4.

Source: https://kjarni.ai/blog/sentence-transformers-without-pytorch/ (published 2026-11-05)

`pip install sentence-transformers` brings PyTorch with it. On a CPU-only server that is
1.4 GB of packages to compute 384 numbers per sentence. It is fine on a workstation and a
problem in a Docker image, an AWS Lambda function or a CI job that installs it every run.

The same model runs without any of it:

```bash
pip install kjarni
```

```python
from kjarni import Embedder

model = Embedder("minilm-l6-v2")

docs = [
    "How do I reset my password?",
    "Where can I track my order?",
    "What is your refund policy?",
    "How do I update my billing address?",
    "Do you ship internationally?",
]
query = "I need to change my login credentials"

for doc in docs:
    print(f"{model.similarity(query, doc):7.4f}  {doc}")
```

```
 0.5981  How do I reset my password?
 0.0767  Where can I track my order?
-0.0027  What is your refund policy?
 0.4067  How do I update my billing address?
-0.0451  Do you ship internationally?
```

Those are the scores sentence-transformers gives for the same sentences, to all four
decimals. The first run downloads the model (90 MB) from Hugging Face and caches it.

## The same model, side by side

The sentence-transformers version, for comparison:

```python
from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2", device="cpu")
q = model.encode("I need to change my login credentials")
for doc in docs:
    print(f"{float(util.cos_sim(q, model.encode(doc))):7.4f}  {doc}")
```

It prints the same five numbers. Measured in two fresh virtual environments on Python
3.13, Linux x86-64, with PyTorch from its CPU-only index:

| | kjarni 0.4.2 | sentence-transformers 6.2 + torch 2.15 (CPU) |
|---|---|---|
| Installed size, venv included | 35 MB | 1.4 GB |
| Packages installed | 1 | 42 |
| Import | 0.02 s | 2.4 s |
| Load the model | 0.05 s | 1.0 s |
| Encode 1,000 sentences | 0.97 s | 0.78 s |

An empty virtual environment is 12 MB, so Kjarni adds 23 MB. The PyTorch figure is the
CPU-only build; the default PyTorch wheel on Linux adds the CUDA libraries on top.

The last row goes the other way: PyTorch encodes a large batch about 20% faster here. If
you embed millions of sentences in one long-running job, that matters more than start-up
time. For a web service, a serverless function, a CLI or a CI step, where the process
starts often and encodes little each time, the 3.4 seconds of import and load usually
matter more.

## Batches and numpy

```python
vectors = model.encode_batch(["first sentence", "second sentence"])
print(vectors.shape)   # (2, 384)
```

`encode_batch` returns a numpy array when numpy is installed, and a list of lists when it
is not. Kjarni does not depend on numpy, so it is never installed for you. Vectors are
L2-normalized by default, so a dot product is the cosine similarity; pass
`Embedder("minilm-l6-v2", normalize=False)` for the raw vectors.

## In a Docker image

```dockerfile
FROM python:3.12-slim
RUN pip install --no-cache-dir kjarni
# Download the model at build time, so containers start offline.
RUN python -c "from kjarni import Embedder; Embedder('minilm-l6-v2')"
COPY app.py .
CMD ["python", "app.py"]
```

With an `app.py` of:

```python
from kjarni import Embedder

model = Embedder("minilm-l6-v2")
score = model.similarity("I need to change my login credentials", "How do I reset my password?")
print(f"{score:.4f}")
```

```bash
docker build -t embed-test .
docker run --rm --network none embed-test
```

```
0.5981
```

`--network none` shows the container needs nothing from the internet once built. The
image is 243 MB, base image and model included.

The wheel holds a native library built for glibc on Linux x86-64 and arm64, so the
`-slim` images work. Alpine uses musl and does not.

## What else it runs

The same `Embedder` loads other sentence-transformers models by name: `mpnet-base-v2`
(768 dimensions, better quality, slower), `multilingual-e5-small` and `bge-m3` for other
languages, and `nomic-embed-text` for longer inputs. They download on first use like
MiniLM. Models load by their name in Kjarni's list, not by an arbitrary Hugging Face
repository id.

The package also has `Classifier`, `Reranker`, `Indexer` and `Searcher` for search,
`Chat` for local LLMs and `Transcriber` for Whisper, all without PyTorch.

## What it does not do

- **No fine-tuning.** Kjarni runs models; it does not train them.
- **Inputs are cut at 256 tokens, the same as sentence-transformers.** About 900
  characters of English. See [all-MiniLM-L6-v2 Max Sequence Length](/blog/embedding-truncation/)
  for what that does to long documents and how to chunk.
- **No musl.** Alpine needs the glibc image or a different base.

## FAQ

### Can I use all-MiniLM-L6-v2 without PyTorch?

Yes. `pip install kjarni` runs it from a native library, with no PyTorch, no
transformers and no ONNX export, and gives the same similarity scores as
sentence-transformers to four decimals.

### How much smaller is it than sentence-transformers?

23 MB installed against 1.4 GB for sentence-transformers with CPU-only PyTorch, measured
in fresh virtual environments. The CUDA build of PyTorch is larger still.

### Is it faster?

It starts much faster: 0.07 seconds to import and load, against about 3.4 seconds. On a
large batch PyTorch is about 20% faster on the same machine.

### Does it need a GPU?

No. It runs on the CPU. `Embedder("minilm-l6-v2", device="gpu")` uses a GPU through
Vulkan, Metal or DirectX 12 when one is present.

## Getting it

```
Install: pip install kjarni
PyPI:    https://pypi.org/project/kjarni/
GitHub:  https://github.com/olafurjohannsson/kjarni
Example: https://github.com/kjarni-ai/kjarni-examples/tree/main/python-semantic-search
```

## Related

- [Local Semantic Search in Python](/blog/semanticsearchpython/): index a folder and search it by keyword, meaning or both
- [all-MiniLM-L6-v2 Max Sequence Length](/blog/embedding-truncation/): 256 tokens, not 512
- [Benchmarking against PyTorch](/blog/benchmarking-against-pytorch/): how the outputs are checked