# Image Search in C#: Find Photos by Description with CLIP, Offline

> Search a folder of photos by typing what is in them, from C#, with no cloud API, no Python and no ONNX export. CLIP runs in your process from one NuGet package. Tested on 1,000 COCO photos: 19 ms per photo to index, and the right photo in the top 5 for 85% of captions.

Source: https://kjarni.ai/blog/imagesearch/ (published 2026-11-03)

Type "a dog on a beach" and get back the photos of a dog on a beach, from a folder nobody
has tagged. In C#, on your own machine, with one NuGet package.

```bash
dotnet add package Kjarni
```

```csharp
using Kjarni;

using var index = new ImageIndex();
index.AddDirectory("/home/me/Pictures");

foreach (var hit in index.Search("a dog on a beach", 5))
    Console.WriteLine($"{hit.Score:F3}  {hit.Path}");
```

The first run downloads the CLIP model (580 MB) and caches it. After that it works with no
network. Every number in this post comes from running this code, Kjarni 0.4.2 from NuGet,
on 1,000 photos from the COCO dataset.

## Try it on your own photos

You need the [.NET 8 SDK](https://dotnet.microsoft.com/download). Then:

```bash
dotnet new console -o PhotoSearch
cd PhotoSearch
dotnet add package Kjarni
```

Replace `Program.cs` with:

```csharp
using Kjarni;

var folder = args.Length > 0 ? args[0] : Environment.GetFolderPath(Environment.SpecialFolder.MyPictures);
var indexFile = Path.Combine(folder, ".photos.kjimg");

using var index = new ImageIndex();
if (File.Exists(indexFile))
{
    Console.WriteLine($"Loaded {index.Load(indexFile)} photos from {indexFile}");
}
else
{
    Console.WriteLine($"Indexing {folder}, this takes a moment the first time...");
    var report = index.AddDirectory(folder);
    index.Save(indexFile);
    Console.WriteLine($"Indexed {report.Added} photos, skipped {report.Skipped}");
}

while (true)
{
    Console.Write("\nSearch: ");
    var query = Console.ReadLine();
    if (string.IsNullOrWhiteSpace(query)) break;

    foreach (var hit in index.Search(query, 5))
        Console.WriteLine($"  {hit.Score:F3}  {hit.Path}");
}
```

And run it on a folder of photos, or with no argument to use your Pictures folder. Here it
is on the COCO photos used in this post:

```bash
dotnet run -- ~/coco-photos
```

```
Indexing /home/me/coco-photos, this takes a moment the first time...
Indexed 1000 photos, skipped 0

Search: a dog on a beach
  0.311  /home/me/coco-photos/000000052891.jpg
  0.227  /home/me/coco-photos/000000050326.jpg
  ...
```

The index is saved beside the photos as `.photos.kjimg`, so the next run starts searching
at once. Delete that file to index the folder again. An empty line quits.

## The other ways to do it

- **A cloud vision API** tags or captions each photo for you. Every photo is uploaded, you
  pay per image, and the search only knows the tags the service chose.
- **A Python service** with PyTorch and Faiss next to your .NET app. It works, but you now
  deploy and run a second runtime.
- **CLIP exported to ONNX**, run with ONNX Runtime. You still write the query tokenizer,
  the image resize, crop and normalisation, and the search yourself, and they have to
  match how the model was trained.

`ImageIndex` does all of that in your process: it decodes JPEG and PNG, prepares each image
the way CLIP expects, tokenizes the query and ranks by cosine similarity.

## How CLIP finds a photo from a sentence

CLIP is two encoders trained together on hundreds of millions of image and caption pairs:
one turns an image into a vector, the other turns text into a vector of the same size. They
were trained so that a photo and a sentence describing it end up pointing in nearly the same
direction.

```
photo  -> resize, crop 224x224 -> vision encoder -> 512 numbers ┐
                                                                ├─ cosine similarity
"a dog on a beach" -> tokenizer -> text encoder -> 512 numbers  ┘
```

So the photos are embedded once, when you index them, and only the query is embedded at
search time. Nothing reads filenames, EXIF tags or captions. The pixels are all it uses.

## Indexing a folder

```csharp
using var index = new ImageIndex();

var report = index.AddDirectory(photosDir);
Console.WriteLine($"indexed {report.Added}, skipped {report.Skipped}");

index.Save("photos.kjimg");
```

`AddDirectory` walks subfolders, indexes every JPEG and PNG, and returns how many it added
and how many it could not read. A corrupt file is skipped and counted, not thrown, so one
broken download does not stop a scan of the whole library.

On an Intel i7-13700 (24 threads), with the model already cached:

| Step | Time |
|---|---|
| Load the model | 279 ms |
| Index 1,000 photos | 19.3 s, about 19 ms per photo |
| Save the index | 18 ms, 6.2 MB on disk |
| Load the saved index | 14 ms |
| Peak memory, model and index | about 900 MB |

Indexing uses every core: photos are decoded and embedded in parallel. Run it on a
background thread in a desktop app, because a large library takes minutes.

`Save` writes the vectors and paths, not the images. Next time, `Load` the file instead of
re-embedding everything:

```csharp
using var index = new ImageIndex();
if (File.Exists("photos.kjimg"))
    index.Load("photos.kjimg");
else
    index.AddDirectory(photosDir);
```

To add one new photo, call `index.Add(path)`. Adding the same path again replaces its
vector rather than adding a duplicate.

## Searching

```csharp
foreach (var hit in index.Search("a red double decker bus", 3))
    Console.WriteLine($"{hit.Score:F3}  {Path.GetFileName(hit.Path)}");
```

Here are the top results for a few queries. The description column is what a human wrote
about that photo in COCO's annotations. The search never saw those captions; they are here
so you can tell whether the hit is right.

| Query | Score | What the photo shows |
|---|---|---|
| a dog on a beach | 0.311 | "A shaggy dog is at the beach with a frisbee." |
| someone surfing | 0.333 | "A man in black wetsuit on white surfboard riding wave." |
| a birthday cake with candles | 0.292 | "A hand holding a knife and about to cut a large birthday cake." |
| a red double decker bus | 0.339 | "The red, double decker bus is driving past other buses." |
| snow covered mountains | 0.261 | "A group of skiers have left a trail through the snow on the mountain." |
| a cat sleeping on a laptop | 0.352 | "An orange cat rests on a white counter top." |
| | 0.351 | "A kitten lies down on the keyboard of a laptop." |
| | 0.351 | "A cat resting on an open laptop computer." |

The last query shows what to expect when it misses. The top hit is a sleeping cat, but not
on a laptop. The two right answers are one thousandth behind it. Show users several results,
not just the first.

### What the scores mean

The scores are cosine similarities, and with CLIP they are small even for a perfect match:
the hits above are all between 0.26 and 0.36. The order matters, but a score does not tell you
how confident to be. Do not set a cut-off like "only show results above 0.5"; nothing will
pass. If you need to hide weak results, compare each score with the top score of the same
query rather than with a fixed number.

## How good is it?

Picking queries that work is easy. To measure it, I used COCO's own captions: for each of
the 1,000 photos, search with its first human caption and see where that photo ranks among
all 1,000 indexed.

| Photo ranked | Share of captions |
|---|---|
| First | 56.9% |
| In the top 5 | 85.4% |
| In the top 10 | 91.9% |

This is a strict test. A caption like "A person that is going out in the water" fits dozens
of photos in the set, and when another surfer outranks the one it was written about, the test
counts a miss even though a user would be happy. On a larger library the photo you want drops
further down for the same reason, so a results grid of 20 or so works better than a list of 3.

A search over the 1,000 photos took 30 ms on average across the 1,000 searches of this
test.
Most of it is encoding the query with CLIP's text encoder, so it grows slowly with the
library: an index of 100,000 entries searched in about 60 ms.

## What it does not do

- **Only JPEG and PNG.** HEIC from iPhones, WebP, RAW and GIF are not read, and
  `AddDirectory` leaves them out of `Skipped` too, which only counts JPEG and PNG files that
  failed. Compare `Added` with your own count of files, and convert the others first.
- **Black-and-white JPEGs need 0.4.2 or newer.** Earlier versions skip them and count them
  in `Skipped`.
- **No "find photos like this one".** `Search` takes text. There is no image-to-image search
  in the C# API yet.
- **English queries.** This CLIP model was trained on English captions, so write queries in
  English.
- **No reading of text in images.** CLIP sometimes matches a word written in a photo, but
  do not rely on it to find a receipt by its total or a sign by what it says.
- **No removal.** There is no method to take one photo out of the index. If photos are deleted,
  check `File.Exists(hit.Path)` before showing a hit, or rebuild the index.

Searching is safe from several threads at once. Adding is not: do not call `Add` or
`AddDirectory` while another thread searches.

## FAQ

### Does it need an internet connection?

Only the first time, to download the model. After that, indexing and searching run offline,
and no photo or query leaves the machine.

### Does it need a GPU or Python?

No. It runs on the CPU, inside your .NET process. The NuGet package holds the native
library for Windows x64, Linux x64 and arm64, and macOS on Apple Silicon.

### Which .NET versions does it work with?

The package targets .NET 8 and .NET Standard 2.1. These numbers were measured on .NET 8.

### How many photos can it handle?

I tested a saved index of 100,000 entries. It loaded in 1.4 s, a search took about 60 ms,
and the process peaked at 1.5 GB of memory. The saved file is large, 690 MB, because the
vectors are stored as text. Indexing is the slow part: at 19 ms per photo on a 24-thread
desktop, 100,000 photos takes about 32 minutes, once.

### Which CLIP model is it?

LAION's ViT-B/32, trained on LAION-2B (`laion/CLIP-ViT-B-32-laion2B-s34B-b79K`). Pass a
directory to `new ImageIndex(modelDir)` to use another CLIP checkpoint in the same format.

## About the test photos

The photos are the first 1,000 images, by ID, of the COCO 2017 validation set. The
captions are COCO's. The images are Flickr photos under their owners' Creative Commons
licences, which is why this post shows captions rather than the photos themselves. See
[cocodataset.org](https://cocodataset.org/).

## Getting it

```
NuGet:   dotnet add package Kjarni
Package: https://www.nuget.org/packages/Kjarni
GitHub:  https://github.com/olafurjohannsson/kjarni
Docs:    https://kjarni.ai/docs/
```

Kjarni is MIT licensed. Questions and bug reports go to
[GitHub issues](https://github.com/olafurjohannsson/kjarni/issues).

## Related

- [Semantic Search in C#](/blog/semanticsearch/): the same idea for text
- [Document Search in C# without Elasticsearch](/blog/documentsearchengine/): keyword, semantic and hybrid search over files
- [Local Embeddings for Microsoft.Extensions.AI](/blog/embeddinggenerator/): text embeddings behind `IEmbeddingGenerator`
- [ImageIndex reference](/reference/dotnet/kjarni-imageindex/)