kjarni::reranker

High-level text reranking API.

Rerank documents by relevance to a query using cross-encoder models.

Quick Start

use kjarni::reranker;

// One-liner
let ranked = reranker::rerank("minilm-l6-v2-cross-encoder", "query", &docs).await?;

// Builder pattern
let reranker = Reranker::builder("ms-marco-minilm")
    .gpu()
    .build()
    .await?;

let ranked = reranker.rerank("What is rust?", &documents).await?;
let score = reranker.score("query", "document").await?;

RerankOverrides

pub struct RerankOverrides

Overrides for reranking behavior.

Fields

• pub top_k: Option<usize>,: Return only top-k results.
• pub threshold: Option<f32>,: Minimum score threshold.
• pub normalize_scores: bool,: Squash scores to 0..1 with a logistic instead of returning raw logits. Off by default, because a raw logit is what every reference returns: the ms-marco checkpoint declares Identity as its activation, and both transformers and sentence-transformers give the same unsquashed value. Turning this on is a deliberate divergence from torch, and threshold is then read on the 0..1 scale too. Ranking is identical either way, because the squash is monotonic; only the numbers move. To convert a single score without changing the reranker, use [RerankResult::probability].
• pub batch_size: Option<usize>,: Batch size for processing pairs.

for_search

pub fn for_search() -> Self

Create overrides for search reranking (top 10).

for_filtering

pub fn for_filtering(threshold: f32) -> Self

Create overrides for filtering (threshold-based).

top_k

pub fn top_k(k: usize) -> Self

Create overrides for top-k selection.

RerankResult

pub struct RerankResult

A single reranking result.

Fields

• pub index: usize,: Original index in the input documents.
• pub score: f32,: Relevance, higher is more relevant. The cross-encoder's raw logit, which is what torch returns and so runs roughly -11 to +11 rather than 0 to 1. Use [Self::probability] for a readable number, or build the reranker with normalize_scores to get one everywhere. Meaningful as an ordering rather than as a confidence: it is calibrated for ranking, and is only loosely comparable across queries.
• pub document: String,: The document text.

probability

pub fn probability(&self) -> f32

[Self::score] squashed to 0..1, for display.

The score itself stays a logit so it matches torch; this is the readable form. Ordering is unchanged, because the squash is monotonic.

Assumes the default scale. Calling this on a reranker built with normalize_scores squashes twice and is wrong, and no check here can prevent it: a logit of 0.972 is a real value from the ms-marco model, and nothing distinguishes it from a probability of 0.972.

new

pub fn new(index: usize, score: f32, document: impl Into<String>) -> Self

Create a new rerank result.

Reranker

pub struct Reranker

High-level text reranker using cross-encoder models.

Cross-encoders process query-document pairs together, producing more accurate relevance scores than bi-encoder similarity.

builder

pub fn builder(model: impl Into<String>) -> RerankerBuilder

Create a builder for a model.

from_preset

pub fn from_preset(preset: &RerankerPreset) -> RerankerBuilder

Create a builder from a preset.

new

pub async fn new(model: &str) -> RerankerResult<Self>

Create a Reranker with default settings.

Uses CPU, downloads model if needed.

from_path

pub fn from_path(path: impl Into<std::path::PathBuf>) -> RerankerBuilder

Load a reranker from a local path.

Example

let reranker = Reranker::from_path("/models/my-cross-encoder")
    .build()
    .await?;

score

pub async fn score(&self, query: &str, document: &str) -> RerankerResult<f32>

Score a single query-document pair. Relevance of one document to one query, on the same scale as [RerankResult::score].

A raw logit unless the reranker was built with normalize_scores. This has to agree with rerank: the two once disagreed, so scoring a pair directly gave a logit while ranking the same pair gave a probability, and nothing said which you had.

score_pairs

pub async fn score_pairs(&self, pairs: &[(&str, &str)]) -> RerankerResult<Vec<f32>>

Score multiple query-document pairs. Score multiple query-document pairs, on the same scale as [Self::score].

rerank

pub async fn rerank(&self, query: &str, documents: &[&str]) -> RerankerResult<Vec<RerankResult>>

Rerank documents by relevance to a query.

rerank_with_config

pub async fn rerank_with_config(&self, query: &str, documents: &[&str], overrides: &RerankOverrides) -> RerankerResult<Vec<RerankResult>>

Rerank with custom overrides.

rerank_top_k

pub async fn rerank_top_k(&self, query: &str, documents: &[&str], k: usize) -> RerankerResult<Vec<RerankResult>>

Rerank and return only top-k results.

rerank_with_threshold

pub async fn rerank_with_threshold(&self, query: &str, documents: &[&str], threshold: f32) -> RerankerResult<Vec<RerankResult>>

Rerank, keeping only results at or above threshold.

threshold is on the same scale as score: 0..1 by default, or raw logits if return_raw_scores is set.

rerank_owned

pub async fn rerank_owned(&self, query: &str, documents: &[String]) -> RerankerResult<Vec<RerankResult>>

Rerank

model_id

pub fn model_id(&self) -> &str

Get the model identifier.

model_type

pub fn model_type(&self) -> Option<ModelType>

Get the model type (None if loaded from path).

model_name

pub fn model_name(&self) -> &str

Get the model's CLI name (if from registry).

device

pub fn device(&self) -> Device

Get the device.

max_seq_length

pub fn max_seq_length(&self) -> usize

Get maximum sequence length.

hidden_size

pub fn hidden_size(&self) -> usize

Get the hidden size.

RerankerBuilder

pub struct RerankerBuilder

Builder for configuring and constructing a Reranker.

new

pub fn new(model: impl Into<String>) -> Self

Create a new builder for the specified model.

from_preset

pub fn from_preset(preset: &RerankerPreset) -> Self

Create a builder from a preset.

cpu

pub fn cpu(mut self) -> Self

Run on CPU (default).

gpu

pub fn gpu(mut self) -> Self

Run on GPU.

auto_device

pub fn auto_device(mut self) -> Self

Auto-select device.

with_context

pub fn with_context(mut self, context: Arc<WgpuContext>) -> Self

Provide a pre-created WgpuContext.

cache_dir

pub fn cache_dir(mut self, path: impl Into<PathBuf>) -> Self

Set custom cache directory.

model_path

pub fn model_path(mut self, path: impl Into<PathBuf>) -> Self

Load model from a local path.

load_config

pub fn load_config(mut self, config: LoadConfig) -> Self

Set model loading configuration.

with_load_config

pub fn with_load_config<F>(mut self, f: F) -> Self where F: FnOnce(LoadConfigBuilder) -> LoadConfigBuilder,

Configure loading with a builder.

download_policy

pub fn download_policy(mut self, policy: DownloadPolicy) -> Self

Set download policy.

offline

pub fn offline(mut self) -> Self

Never download models.

top_k

pub fn top_k(mut self, k: usize) -> Self

Set default top-k results to return.

threshold

pub fn threshold(mut self, threshold: f32) -> Self

Set minimum score threshold.

normalize_scores

pub fn normalize_scores(mut self, normalize: bool) -> Self

Squash scores to 0..1 instead of returning raw cross-encoder logits.

A deliberate divergence from torch, which returns the logit. Ranking is unaffected either way since the squash is monotonic; threshold is read on whichever scale is in use. For a single readable number without changing the reranker, use RerankResult::probability().

overrides

pub fn overrides(mut self, overrides: RerankOverrides) -> Self

Apply full overrides.

quiet

pub fn quiet(mut self, quiet: bool) -> Self

Suppress non-error output.

build

pub async fn build(self) -> RerankerResult<Reranker>

Build the Reranker.

RerankerPreset

pub struct RerankerPreset

A reranking model preset.

Fields

• pub name: &'static str,: Preset name.
• pub model: &'static str,: CLI model name.
• pub max_seq_length: usize,: Maximum sequence length (query + document).
• pub recommended_device: KjarniDevice,: Recommended device.
• pub memory_mb: usize,: Approximate memory in MB.
• pub description: &'static str,: Description.

RerankerError

pub enum RerankerError

Errors specific to reranking.

Variants

• UnknownModel(String),: Model not found.
• IncompatibleModel: Model cannot rerank.
• ModelPathNotFound(String),: Model path not found.
• ModelNotDownloaded(String),: Model not downloaded.
• DownloadFailed: Download failed.
• LoadFailed: Load failed.
• RerankingFailed(#[from] anyhow::Error),: Reranking failed.
• GpuUnavailable,: GPU unavailable.
• InvalidConfig(String),: Invalid configuration.

RerankerResult

pub type RerankerResult<T> = Result<T, RerankerError>

Result type for reranker operations.

RerankerTier

pub enum RerankerTier

Tier-based preset selection.

Variants

• Fast,: Smallest, fastest.
• Balanced,: Balanced quality/speed.
• Quality,: Highest quality.

resolve

pub fn resolve(&self) -> &'static RerankerPreset

Resolve tier to default preset.

available_models

pub fn available_models() -> Vec<&'static str>

List all available reranking models.

is_reranking_model

pub fn is_reranking_model(model: &str) -> RerankerResult<()>

Check if a model is valid for reranking.

rerank

pub async fn rerank(model: &str, query: &str, documents: &[&str]) -> RerankerResult<Vec<RerankResult>>

Rerank documents by relevance to a query.

Returns documents sorted by relevance score (highest first).

Example

let documents = vec!["doc1", "doc2", "doc3"];
let ranked = kjarni::reranker::rerank("ms-marco-minilm", "query", &documents).await?;
for (idx, score) in ranked {
    println!("Doc {}: {:.4}", idx, score);
}

score

pub async fn score(model: &str, query: &str, document: &str) -> RerankerResult<f32>

Score a single query-document pair.

Example

let score = kjarni::reranker::score("ms-marco-minilm", "What is Rust?", "Rust is a programming language.").await?;
println!("Relevance: {:.4}", score);

reranker::presets

Versioned presets for reranking models.

ALL_V1_PRESETS

pub const ALL_V1_PRESETS: &[&RerankerPreset] = &[&RERANKER_MINILM_V1, &RERANKER_MSMARCO_V1]

All V1 presets.

RERANKER_MINILM_V1

pub const RERANKER_MINILM_V1: RerankerPreset = RerankerPreset

MiniLM cross-encoder - small and fast.

RERANKER_MSMARCO_V1

pub const RERANKER_MSMARCO_V1: RerankerPreset = RerankerPreset

MS MARCO MiniLM - optimized for passage retrieval.

find_preset

pub fn find_preset(name: &str) -> Option<&'static RerankerPreset>

Find a preset by name.