ArchiveAgent
Can a small model name the house, season and year of a runway photo?
Theia is a fine-tune of Kimi K2.6, an open model from Moonshot AI. It was trained on Tinker with 5,025 examples drawn from 7,512 photos of 14 houses: one pass, 80 minutes, $18.35. Every model below got the same photos and the same instructions. A decline counted as wrong.
One run per model, scored by us, with model-made labels. We found and fixed three mistakes in our own scoring; they are listed on archiveagent.si.Four tests
| Test | Theia | Best other | Who |
|---|---|---|---|
| Runway, 287 photos | 56.4% | 44.9% | Kimi K3 |
| Product house, 150 photos | 100% | 98.7% | Claude Opus 5.5 |
| Blind runway, 50 photos | 56% | 26% | Gemini 3.1 Pro |
| Exact product, 75 products | 50.7% | 58.7% | Claude Opus 5.5 |
Theia costs about 0.14 cents per photo. Per photo it gets fully right, that is about 17 times less than Claude Opus 5.5.
Where it falls short
It loses the exact product test, and with 75 products the gap could be luck. On houses it never trained on, it names the house 57% of the time, against 90% on the 14 it knows. Part of its edge on made-up product names comes from having seen those product lines in training. It always answers, even when unsure, and on studio product photos the season and year are mostly guesses.
Data and code
The test photos, every model’s answers and the scores are on Hugging Face. So are the tests run on a home computer, without photos. The photos are under copyright, so you have to ask for access. The link is on archiveagent.si.