# Reranking in C# with a Cross-Encoder

> Improve search relevance in .NET with cross-encoder reranking. Rescore your existing search results from Elasticsearch, SQL LIKE, or vector search, locally in C# and in your own process.

Source: https://kjarni.ai/blog/reranking/ (published 2026-08-27)

Your search works. Someone types a question, twenty results come back, and the one they
wanted is at position eleven.

That's not a broken index. It's the gap between **recall** and **precision**. Retrieval is
built for recall: get the right document *somewhere* in the top twenty, fast, across
millions of candidates. But people read the top three, and a RAG prompt only has room for the
top three. Precision at the very top is a different problem, and it needs a different tool.

That tool is a cross-encoder.

```cs
using Kjarni;

using var reranker = new Reranker(quiet: true);

var results = reranker.Rerank(
    "how do I cancel my subscription",
    [
        "Our office is open until 5pm on weekdays.",
        "To end your plan, go to Settings and choose Close account.",
        "Cancellation of orders is handled by the warehouse team.",
        "You can update your billing address at any time.",
    ]);

foreach (var r in results)
    Console.WriteLine($"{r.Score:F3}  {r.Document}");
```

```bash
dotnet add package Kjarni
```

The winner is the sentence that never says "cancel" or "subscription": *"To end your plan,
go to Settings and choose Close account."* The decoy that repeats both words, "Cancellation
of orders is handled by the warehouse team", ranks below it. Keyword search gets this exactly
backwards.

The cross-encoder is about 88MB and downloads on first use. Everything after that runs on
your CPU, offline.

## Bi-encoders and cross-encoders

Worth understanding, because it explains both why reranking works and why you can't just use
it for everything.

A **bi-encoder**, which is what ordinary embedding search uses, encodes the query and each
document *separately* into vectors, then compares them. The document vectors are computed once, ahead
of time, so querying a million documents is a million cheap vector comparisons. Fast. But
query and document never actually meet: the model compresses each into a fixed vector before
any comparison happens, and nuance is lost in that compression.

A **cross-encoder** feeds the query and the document through the model *together*, as a pair,
and outputs a single relevance score. The model sees both texts at once and can weigh the
specific words in the query against the specific words in the document. Much more accurate,
and much slower, because nothing can be precomputed. Every pair is a fresh forward pass.

So you use both, each where it's strong:

| | Bi-encoder | Cross-encoder |
| --- | --- | --- |
| Compares | vector to vector | query and document together |
| Precomputable | yes | no |
| Speed | millions of docs | tens to hundreds |
| Accuracy at the top | good | better |
| Use for | retrieval | reranking |

**Retrieve broadly, rerank narrowly.** Pull 50 candidates with whatever search you already
have, then rerank those 50 and keep the best 5.

## Reranking what you already have

The most useful thing about a standalone reranker is that it doesn't care where the
candidates came from. Elasticsearch, Postgres full-text, a `LIKE` query, a vector database,
a hand-written filter. If you can produce candidate strings, you can rerank them:

```cs
// However you already search:
List<Document> candidates = await _elastic.Search(query, size: 50);

using var reranker = new Reranker(quiet: true);

var ranked = reranker.RerankTopK(
    query,
    candidates.Select(c => c.Body).ToArray(),
    k: 5);

// RerankResult.Index points back into the array you passed in,
// so you can recover the original object with its metadata.
var best = ranked.Select(r => candidates[r.Index]).ToList();
```

`RerankTopK` is the one to reach for: it scores everything and returns only the best `k`,
which is what you almost always want.

One thing to watch when the candidates come from somewhere else: the cross-encoder reads
the query and the document *as one sequence*, and that sequence is capped at 512 tokens,
roughly 1,800 characters shared between the two. A long document gets truncated from the
end, and the query eats into the same budget. If your search returns whole pages, rerank
the passage you would actually put in the prompt rather than the full document. The same
trap in its bi-encoder form is measured in
[the 256-token limit in all-MiniLM-L6-v2](/blog/embedding-truncation/).

That `Index` field matters. Reranking gives you back a reordering, not new documents, so you
map the indices onto your own objects and keep IDs, URLs, permissions and timestamps intact.

## Scoring a single pair

Sometimes you don't want a ranking, you want a number:

```cs
using var reranker = new Reranker(quiet: true);

float score = reranker.Score(
    "capital of Iceland",
    "Reykjavik is the capital and largest city of Iceland.");

Console.WriteLine(score);   // 8.55
```

**These are raw logits, not probabilities.** They are unbounded and frequently negative, which
surprises people the first time they see it. A score of `-5` does not mean "no match". Only
the ordering is meaningful, and the absolute value only means something relative to other
scores from the same model.

Here is the actual range, measured with `minilm-l6-v2-cross-encoder`:

| Score | Query | Document |
| ----- | ----- | -------- |
| **+8.55** | capital of Iceland | Reykjavik is the capital and largest city of Iceland. |
| **−4.96** | what is the refund window | Refunds are available within 30 days of purchase. |
| **−5.45** | how do I cancel my subscription | To end your plan, go to Settings and choose Close account. |
| **−8.19** | how do I cancel my subscription | Cancellation of orders is handled by the warehouse team. |
| **−10.92** | what is the refund window | Returns are accepted within 30 days of delivery. |
| **−11.12** | capital of Iceland | Bananas are a good source of potassium. |
| **−11.23** | how do I cancel my subscription | Our office is open until 5pm on weekdays. |

Two things worth noticing. A direct factual hit scores far above everything else, and total
irrelevance clusters around −11, so there *is* a usable signal for thresholding. But look at
the refund pair: "Returns are accepted within 30 days of delivery" is a reasonable answer to
"what is the refund window" and still scores −10.92, because the model does not treat
*returns* and *refunds* as the same thing. A threshold tuned on the first example would
discard the second.

That's the lesson: thresholding works, but **calibrate it on your own queries and your own
documents**. No threshold anyone can quote you will transfer to your corpus. Run fifty real
queries, look at where the good and bad answers actually fall, and put the cutoff between
them.

If you'd rather work in 0 to 1, apply a sigmoid, `1f / (1f + MathF.Exp(-score))`, but that is
a presentation change, not a calibration. It moves the same ordering onto a friendlier scale.

## Built into search

If you're using Kjarni's `Searcher`, reranking is a constructor argument and a flag. The
two-stage retrieval happens for you:

```cs
using var searcher = new Searcher(
    model: "minilm-l6-v2",
    rerankerModel: "minilm-l6-v2-cross-encoder",
    quiet: true);

var results = searcher.Search("docs.idx", "rotate the signing key",
    topK: 5,
    rerank: true);
```

It retrieves a wider candidate set internally, rescores with the cross-encoder, and returns
the top 5. See [RAG in C#](/blog/ragincsharp/) for where this sits in a full pipeline.

## What it costs

Reranking is a forward pass per candidate, so cost scales linearly with how many you rerank.
That's the entire performance model, and it leads to one rule: **rerank tens, not thousands.**

Retrieve 50 and rerank 50. Retrieving 1,000 and reranking all of them abandons the reason the
two-stage design exists.

If latency is tight, rerank fewer candidates before you reach for a smaller model. Going
from 100 to 30 candidates is a much larger saving than any model swap, and usually costs less
accuracy.

## When it isn't worth it

Reranking is not free and not always the answer.

- **Your retrieval returns nothing relevant.** Reranking reorders candidates; it cannot
  invent one. If the right document isn't in the top 50, fix retrieval first: widen the
  candidate set, or switch to hybrid search so exact identifiers match.
- **Your corpus is tiny.** With 20 documents total, embed and compare them all directly.
- **Your queries are exact lookups.** Order numbers and SKUs want an index, not a language
  model.

The clearest signal that reranking will help: the right answer is reliably *in* your results,
just not at the top. That's precisely the gap it closes.

## FAQ

### What is a cross-encoder?

A model that takes a query and a document *together* and outputs a relevance score. Unlike
embedding search, which compares two separately-computed vectors, a cross-encoder sees both
texts at once. More accurate, and too slow to run over a whole corpus.

### Do I need to replace my existing search?

No. Reranking sits on top of whatever you already use: Elasticsearch, Postgres, a vector
database, or plain SQL. You hand it the candidates and it reorders them.

### How many results should I rerank?

Tens. Retrieve 50 and rerank 50 is a good default. Cost is linear in candidate count.

### Does this need Python or an API key?

No. The NuGet package ships a native library that runs inside your process: nothing to
install alongside it, nothing to start, and no network call.

### Does it work offline?

Yes. The model downloads once on first use and caches locally.

### Can I use my own threshold to filter weak results?

Yes. Use `Score(query, document)` and calibrate a cutoff against your own data. Scores are
relative to the model rather than absolute probabilities.

### What does it cost?

Nothing. MIT or Apache-2.0 licensed, no API key, no per-query billing.

## Try it

```bash
dotnet add package Kjarni
```

Source and issues: [github.com/olafurjohannsson/kjarni](https://github.com/olafurjohannsson/kjarni)
