Search by picture · 2025
Image Retrieval System
You give it one photo and it ranks 10,000 others by how similar they look. It uses no labels and no deep learning, just a careful description of each image and a way of measuring how far apart two descriptions are.
78% top-5 recall on 10,000 images
Step 1 · describe
Describing an image
A computer can't compare two photos directly. So I cut each image into a grid and reduced every cell to its average colour. Put those colours one after another and the photo becomes a list of numbers, called a descriptor.
Two photos with similar descriptors have similar colours in similar places. That is what makes it work, and also its biggest weakness.
Step 2 · compare
Measuring similarity
The same descriptors give different results depending on how you measure the distance between them, so I tried several and compared them.
L1 distance
Adds up the differences number by number, like walking around city blocks. Simple and fast.
Cosine similarity
Compares the direction of the two lists and ignores their size, so a darker copy of a photo still matches.
Mahalanobis distance
Takes into account how the whole collection varies, so an unusual difference counts for more than a common one.
Step 3 · look at what comes back
Results
query WorkedMahalanobis distance
The first five results are sheep on grass. Further down the list it drifts to cows, which share the same green field and pale body.
query Mostly workedL1 distance
Eight of the ten are trees against sky. A wall sign and a goose slip in, because their colour layout happens to be close.
query FailedThe failure case
After the bicycle itself come a dog, a bench, faces and a bookshelf. This showed me the limit of the method: it matched the brown and grey backgrounds, not the idea of a bicycle.
Results are shown exactly as the system returned them, best match on the left. The photos come from the research image set the project was tested on.
Design
System design
- Extractorturns each image into a descriptor
- Indexstores all 10,000 descriptors
- Rankermeasures distance and sorts
- Python
- MATLAB
I built the extractor, index and ranker as separate parts. That meant I could change the descriptor without touching the ranking, or swap the distance measure without rebuilding the index, and test each change on its own.
A content-based image retrieval system achieving 78% top-5 recall on a 10,000-image dataset, using descriptor-based feature extraction and cosine similarity ranking.
- Feature extractor, index and ranker built as independent components, allowing isolated parameter tuning at each stage