Every species name sits as a vector a photo can be held up against. The index took four hours to build and is only rebuilt when the taxonomy moves.
We run the plant recognition on our own GPU
PlantGeekz can put a name on a plant photo without asking anyone outside. The model sits on a server we own ourselves and works on a catalogue of some 860,000 accepted plant names. Here is what that gives us that an outside service does not.
FIG. 01
Scale
What sits behind an answer
The short version: a photo goes in, and an answer comes out at three levels. Family, genus and species, each with its own confidence. None of it happens at a third party. The model is BioCLIP 2, it is publicly available under an MIT licence, and it runs in a container on a Tesla T4 on our own server alongside the rest of PlantGeekz.
Here we have real images alongside the name. That is what lifts the species guess from a third to around three quarters.
Not just the species but the exact cultivar. A model that only knows names cannot do that.
These are read off the running service, not worked out for the occasion. Two of them only ever go up.
FIG. 02
The answer
It answers at three levels instead of one
The obvious thing would be to ask for a single species name. We measured what that would cost us: on images the model had never seen, with every species in play, a plain species guess is right a little over a third of the time. Ask for the genus instead and it gets two out of three. Ask for the family and it gets close to eight out of ten.
So a single species name would be wrong far more often than right, and the user would have no way of telling the two apart. That is why the service answers at all three levels and lets the app show what it is willing to stand behind. A confident family is worth more than a wrong species.
On top of that sits a layer that does not guess but recognises. If we already have photos of that exact cultivar, it compares the picture directly against them. When one of them stands clearly apart from the rest, it answers with the full cultivar name. When two cultivars look too much alike, it says nothing and lets the species answer stand. That behaviour is what makes it trustworthy on the occasions where it does get specific.
One photo
- Photo
- Family
- Genus
- Species
- This exact cultivar
Every step has its own confidence. The app shows the ones that hold and skips the rest.
FIG. 03
The advantages
Why it lives here rather than somewhere else
Sending the images to a finished service would have been quicker. Here are the six reasons we did not.
The price does not follow the usage
A lookup at an outside service costs money every single time. Ours costs electricity on a machine we already have. Whether a thousand or a hundred thousand images go through in a month barely moves the bill. It also means we can let people try again without watching a counter.
The photos stay where they are
User photos never leave our own server. There is nobody to ask about what they store, how long they keep it, or whether it ends up as training data somewhere we cannot see.
Nobody can switch it off
The model is open and sits on our disk. Nobody can raise the price from one month to the next, change the terms, or retire an endpoint we built a product on top of. That is a risk you only notice on the day it fires.
We can measure it
We can hold a change up against exactly the same images as last time and see whether the number actually went up. You cannot do that when the answer comes from a black box that gets updated without notice. Our weighting between name and photo was chosen because it won a measurement, not because it sounded right.
It knows our plants specifically
Large models know species. They do not know the thousands of cultivars people actually grow. A common hazel and a purple-leaved hazel share a species name and look nothing alike. Our layer can tell them apart because it works on photos of the exact plants rather than on names.
It sharpens itself
Every approved user photo makes the recognition better the next morning. We do not buy a new version, and we do not retrain anything. We simply have more examples today than we had yesterday.
FIG. 04
The loop
It learns overnight, without anyone training anything
Someone takes a photo
A user uploads a picture of their own plant and attaches it to the name that plant has in the system.
A human looks at it
No image joins the material before a moderator has approved it. That is the only manual part of the whole loop, and it is there on purpose.
Half past five in the morning
A nightly job folds the new images into the recognition and restarts the service. If nothing new came in, it stops after a second without touching the GPU at all.
Sharper by breakfast
No weights changed, and no model was retrained. It simply has more images to hold the next photo up against. That is also why it takes minutes rather than hours.
FIG. 05
Scope
What it can do, and what it cannot
This kind of thing is quickly sold as more than it is. So both lists are here.
It can
- Put a family, genus and species on a plant photo, each with its own confidence
- Recognise the exact cultivar when we already have photos of it
- Hold back when two cultivars look too much alike for a safe answer
- Keep running even if an outside service is down or shuts up shop
- Get better every night without anyone touching a model
It cannot
- Guess right on a cultivar we have never received a photo of
- Rescue a blurred close-up of a leaf with nothing else to go on
- Identify diseases, pests or deficiencies
- Replace a botanist when the answer really has to hold
The technical part was not the hard part. The hard part was working out what the system should answer when it was not sure, and building it so that it gets better from being used rather than standing still. Those are the choices that decide whether a system like this survives in production or turns into something nobody wants to show off any more.
FIG. 06
Next
More to read
Translation without a token bill
PlantGeekz ships in 16 languages. Some is written with a paid model, the rest translated on our own GPU. Why the cheap model is not a shortcut when the text is full of botanical names.
Read the article
Large data volumes
What happens to a system when the tables grow from a hundred thousand rows to twenty million. What it costs not to have the data model under control, and what I do about it.
Read the article
Got something that needs building properly?
Write a few lines about what you are dealing with. I will come back with what I think, including if the answer is that someone else would do it better.