Here is a sentence someone might type into a photo search:

Show me a photo of Maya sitting next to Tom in a red car on a sunny day.

It reads like one request. It is really about six separate claims, and each one needs a different kind of evidence. Maya has to be in the picture. So does Tom. They have to be next to each other. There has to be a car. It has to be red. The weather has to be sunny.

A search engine that treats the whole sentence as a bag of words will return something for it, because it always returns something. Most of what it returns will be wrong in ways that look plausible. The work on 3 September was about taking that sentence apart and asking, for each part, what would actually count as proof.

First, something to search: a sentence about each photo

Until this work, nothing in NAOMS ever wrote a sentence about what a photo shows. There were labels: a scene tag here, a detected face there, text read out of a sign. A pile of labels can say "outdoor, car, two faces". It cannot say "two people sitting side by side in a red car", because a label list has no way to express a relationship between the things in it.

So each imported photo now gets a short prose description, written by a vision model: a model that takes an image as input and writes text about it. The important word is local. The model runs on your own machine, and the description is stored with the photo on that machine. Each device describes its own copy.

That needed real plumbing, not a setting. Before this change the only model connections that could accept images at all were the two cloud providers. The local ones silently turned a picture into the literal text [image:base64] and sent that instead. Pointing a local text model at a photo would not have failed. It would have made something up.

That is not a hypothetical. When the image data was removed from the request and the same prompt was sent to a capable local vision model, it did not say "I see no image". It described two women sitting on a couch, one of them holding something bright red. The real test picture was a red square on a white background. The invented scene even contained the word "red", so a check that only looked for keywords would have passed it.

Three rules follow from that, and they are the default behaviour:

• A local model is sent an image only if it is known to read images. The list is an allow-list. A model that is not on it is refused, not tried.
• The description step does not send a photo to a cloud model. Photos are marked as staying on your machine, and if the only vision model available is a cloud one, the photo gets no description. As the code puts it, it would rather have no description than an uploaded photo.
• No local vision model means no description. Search still works on everything else. It simply has nothing machine-written to search.

The broader principle, that local is the default and reaching outward needs an explicit grant, is the subject of There is no ungoverned route to a model.

The description is also kept apart from your own caption. If you wrote a caption, it stays yours, byte for byte. The machine's sentence lives in a separate field, so "I wrote this" and "a model guessed this" stay two different kinds of claim. An earlier piece covers why that distinction runs through the whole photo system.

The pipeline was then run end to end on real imports with a real local model. Each test picture carried something only its own pixels could reveal, such as a rendered word or a particular shape-and-colour pairing. The check was that each description matched its own photo, never the other set, and that the same model with no image produced neither.

"A" is not evidence

With descriptions in place, the next problem was the question itself.

Several search methods run side by side, and their results are combined. Some of them match by substring. Under that design, the little words in a sentence stop being harmless. "A" is a substring of almost every place name. "In", "of", "to", "on": the same. In the test sentence, ten of the twenty-one words were filler of this kind. The letter "a" alone matched "Marrakech".

The effect was measured, not guessed at. Against a library holding only one photo, which matched none of the sentence's six claims, the search returned that photo, confidently. The honest answer was nothing.

The fix was to make the two places that split a question into words share one list of words that carry no meaning, instead of each keeping its own. Connecting words no longer count as a match for anything.

"His friend Tom" means Tom

The next claim hid in a phrase like "his friend Tom".

The first version handled "friend" correctly and ignored the name. It found everyone marked as a friend, so a photo of a different friend satisfied the query. The relationship narrowed the results. The name did nothing.

Now the relationship is the gate and the name is the selector. A contact only becomes a candidate by already being a friend. Matching words in the query never makes someone a friend. The name then picks which friend. If you name someone who is a contact but not a friend, the answer is empty. It does not quietly widen back to every friend you have.

"Next to" is measured

"Sitting next to" sounds like a phrase to look for. There was nothing to look for it in. Photos do not come with sentences about who is beside whom, and matching the words against a caption you happened to write would be coincidence dressed up as understanding.

What the system does have is where each detected face sits in the frame. Two boxes are enough to compute adjacency:

• The centres must be close, within about three and a half face-widths of each other. People sitting side by side have their face centres roughly one and a half to three face-widths apart.
• The faces must be similar in size, within a factor of two and a half. A big face and a tiny face are at different distances from the camera: one in the foreground, one far behind.

"Next to" is measured in both directions. The system will not answer "to the left of" from these stored boxes. For some photo orientations the stored coordinates can be flipped relative to what you see on screen. Distance and relative size survive that flip. Left and right do not. Refusing a question its data cannot answer honestly is the whole point.

"In Seattle" finds a neighbourhood

Place names are worked out on your own device from a gazetteer, a table of towns and their coordinates that ships with the app. The town-and-season piece describes how that lookup works. The gap here was simple: a photo taken in a Seattle neighbourhood was labelled with the neighbourhood and the country, and nothing else. "Photos in Seattle" could not find it, however clever the query side got.

Labels now carry the municipality the neighbourhood belongs to. On a measured set of ten Seattle locations, search found four before the change and nine after. The tenth is not a failure. That spot resolves to an unincorporated area outside the city, so declining it is correct. Six places that should not match, including Tacoma, Bellevue and Marseille, still did not.

Two details show how easy it is to get this wrong:

• Region codes are only local. Nice in France and Liège in Belgium share the same short administrative code. Comparing that code alone matched a Belgian photo to "Nice". A municipality is now identified by country, region and code together.
• Size does not decide which town counts. Picking the most populous member works for Seattle and fails for Kyiv, where it would name one of the city's own districts as the town. The rule uses the gazetteer's marker for an administrative seat instead. Where no seat is marked, the label gains nothing.

One limit applies to your existing library: photos imported before this change keep their old labels. Nothing was rewritten in place.

What a photo can prove

The last piece set the ceiling, and it is a rule rather than a search feature yet. The rule says: when two people you know are matched to different faces in the same photo, that photo is evidence they were together. That is pure set membership. There is no threshold and no guesswork, and it is matched on the contact's identity, never on a name. Two people who share a name are not one person.

What the rule refuses to do is the next step. Being photographed together often does not make two people friends. A stranger in the background is a true "together" and a false friendship. So frequency is kept as a count, not turned into a relationship. The commit's title says it plainly: a photo can prove two people were together, and nothing more than that. The rule was written and tested on 3 September; it is not yet called when photos are imported, so it does not yet add anything to your library.

That sentence also describes the whole approach. Each clause of the original question becomes a claim, each claim gets the evidence it actually needs, and where the evidence runs out, the system stops rather than rounding up.

Status

Everything above was built and tested on 3 September. It reached the main line of the codebase on 23 September. The together-in-a-photo rule is the exception noted above: it exists and is tested, but nothing calls it yet. The "sunny day" clause was not part of this work. Descriptions need a local vision model to be installed; without one, photos get none.

Written by AI agents from real project logs; owned and edited by Mujo.