# all-MiniLM-L6-v2 Max Sequence Length: 256 Tokens, Not 512

> all-MiniLM-L6-v2 truncates at 256 tokens, about 900 characters, and drops the rest without a warning. Why the configs say 128, 256 and 512, what chunk size to use, and why raising max_seq_length to 512 makes retrieval worse.

Source: https://kjarni.ai/blog/embedding-truncation/ (published 2026-08-31)

Three files in the model repository disagree about how long an input can be, and
the one that wins is not the one most people assume.

```python
from sentence_transformers import SentenceTransformer
from huggingface_hub import hf_hub_download
import json

m = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
tj = json.load(open(hf_hub_download("sentence-transformers/all-MiniLM-L6-v2",
                                    "tokenizer.json")))

print(tj["truncation"]["max_length"])                       # 128
print(m.max_seq_length)                                     # 256
print(m[0].auto_model.config.max_position_embeddings)       # 512
```

`encode` passes `max_length=self.max_seq_length` on every call, so the tokenizer's
128 gets overridden and the model's 512 is never reached. 256 wins. For
`all-mpnet-base-v2` it is 384.

## It truncates without telling you

Same text, same model, only the window changes:

```python
body = ("The retrieval service keeps an inverted index in memory and refreshes it "
        "nightly from the document store. Queries are served from that snapshot, so "
        "writes are not visible until the next refresh completes. ")
doc = body * 12 + "The rollback procedure is documented under runbook QX-4417."
print(len(m.tokenizer(doc, truncation=False)["input_ids"]))   # 509

m.max_seq_length = 256
a256 = m.encode(doc, normalize_embeddings=True)
m.max_seq_length = 512
a512 = m.encode(doc, normalize_embeddings=True)

print(a256 @ a512)                                            # 0.9359
```

Two different vectors for the same document. No warning, no error, no truncation
flag in the output. At the default 1000-character chunk size this is the normal
case, not an edge case:

```python
lengths = [len(m.tokenizer(c, truncation=False)["input_ids"]) for c in chunks]
print(sum(l > 256 for l in lengths), "/", len(lengths))       # 85 / 98
```

LangChain's RAG tutorial splits with `RecursiveCharacterTextSplitter` at
`chunk_size=1000`, so on that setup nine chunks in ten are longer than the model
reads. The class default, 4000, is worse. English runs
about 3.5 characters per token, so 256 tokens is roughly 900 characters and a
1000-character chunk sits just past the edge.

## A longer window is not the fix

The document above has its distinctive fact in the tail, past the 256 mark. Two
queries, one for the tail and one for the head:

```python
tail = "what is the rollback runbook number"
head = "how often is the inverted index refreshed"

# tail: 256 -> 0.1239   512 -> 0.2049
# head: 256 -> 0.5154   512 -> 0.4457
```

The tail improving is expected. The head getting *worse* is the part that matters:
these models mean-pool, so 250 more tokens of unrelated content drag the centroid
away from whatever the query was about. A longer window buys recall on the tail and
pays for it in precision everywhere else.

## Chunk instead, and score by the best chunk

```python
import numpy as np

qv = m.encode(tail, normalize_embeddings=True)
cv = np.stack([m.encode(c, normalize_embeddings=True) for c in chunks])  # 3 chunks
best = (cv @ qv).max()

# tail: truncated 0.1239   best-chunk 0.4789
# head: truncated 0.5154   best-chunk 0.5058
```

The tail goes from noise to 0.48, more than double what the 512 window gave, and
the head holds. That is the whole argument against reaching for nomic at 8192 or
bge-m3: a chunk that fits under the limit gets the tail *and* keeps its focus,
while a chunk that needs 8192 tokens averages too many things to match any of them
sharply.

So chunk by tokens rather than characters, since code and tables and non-English
text all have different ratios. Target well under the limit, 200 tokens with
overlap never truncates. Then take the maximum similarity across a document's
chunks instead of embedding it whole.

## Checking your own corpus

```python
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")

lengths = [len(tok(c, truncation=False)["input_ids"]) for c in your_chunks]
over = sum(l > 256 for l in lengths)
print(f"{over}/{len(lengths)} chunks exceed 256 tokens")
```

If that is not close to zero, part of your corpus is not in your index and nothing
in your pipeline will tell you.

## 512 is a hard ceiling

There are exactly 512 learned position embeddings in the weights. Ask for more and
a naive implementation has nothing to add for the extra positions, so you get
either an index error or tokens entering attention with no positional information,
which returns a normal looking vector that is quietly wrong.

## What Kjarni does about it

I hit this building [Kjarni](https://kjarni.ai), a native inference engine. Its
encoders read `max_position_embeddings` and truncated at 512. Parity against
PyTorch passed, because every parity test used short sentences: the longest string
in the entire encoder suite was 83 characters. So it matched the reference exactly
right up until an input crossed 256 tokens, which the default chunk size put 87% of
chunks above.

Kjarni now reads `sentence_bert_config.json` when the model ships one, the same
file sentence-transformers reads, so the truncation point matches whatever your
Python pipeline already does. Asking for more positions than the weights hold is an
error rather than a silently wrong vector.

```bash
kjarni embed "your text" --model minilm-l6-v2
```

The wider lesson was about the tests, not the config. A parity suite whose longest
input is 83 characters proves the model loads, and nothing about the case that
actually broke.

## FAQ

### What is the max sequence length of all-MiniLM-L6-v2?

256 tokens. That is `max_seq_length` in `sentence_bert_config.json`, and it is what
sentence-transformers passes to the tokenizer on every `encode` call. The 512 in
`config.json` is how many position embeddings the weights have, and the 128 in
`tokenizer.json` is overridden. Anything past 256 tokens is dropped.

### What chunk size should I use with all-MiniLM-L6-v2?

Around 200 tokens, measured with the model's own tokenizer, with some overlap. In
characters that is roughly 700 for English prose, but code, tables and other
languages tokenize differently, so counting characters is only an estimate. 256
tokens is about 900 characters, which is why the common 1000-character chunk is just
over the limit.

### Does sentence-transformers warn when it truncates?

No. `encode` truncates to `max_seq_length` and returns a normal vector. There is no
warning, error or flag in the output. Count tokens with
`tokenizer(text, truncation=False)` to find out which inputs were cut.

### Can I set max_seq_length to 512?

You can, and it runs, because the weights have 512 positions. It does not help: the
model was trained at shorter lengths and mean-pools every token, so a longer input
averages more topics together. In the measurement above, 512 improved a query about
the tail of a document and made a query about its head worse. Chunking improved both.
The same holds when the setting is changed through fastembed, LangChain or any other
wrapper, since the weights are the same.

### Does KeyBERT truncate at 256 tokens too?

Yes, by default. KeyBERT's default model is `all-MiniLM-L6-v2`, loaded through
sentence-transformers, so a document longer than 256 tokens is embedded from its
first 256 only. Keywords that appear only later in the document are compared against
an embedding that never saw them. Split long documents before extracting keywords.

### What are the limits of other sentence-transformers models?

`all-mpnet-base-v2` reads 384 tokens. `all-MiniLM-L12-v2` and
`paraphrase-multilingual-MiniLM-L12-v2`, the usual pick for German and other
non-English text, read only 128. Check `max_seq_length` in each model's
`sentence_bert_config.json` rather than assuming a model reads as far as its
position embeddings go.

---

Kjarni is a native inference engine for embeddings, search, reranking and chat,
with bindings for [C#](https://www.nuget.org/packages/Kjarni),
[the browser](https://www.npmjs.com/package/kjarni-wasm), Go and Python. The source
is [on GitHub](https://github.com/olafurjohannsson/kjarni), and there is a
[browser demo](https://kjarni.ai/demo/) that needs no install.

## Related

- [Local Embeddings for Microsoft.Extensions.AI](/blog/embeddinggenerator/): using these models from .NET
- [Build a Document Search Engine in C#](/blog/documentsearchengine/): where chunk size shows up
- [Reranking in C#](/blog/reranking/): a cross-encoder reads query and document together, so it avoids the pooling problem above
