AI descriptions
A description turns Person in Front Path Zone into A person in a grey T-shirt walks down the front path toward the street. Fregata Mobile can show two kinds, and you can use either, both, or neither:
| Written by | Needs | Where it appears | |
|---|---|---|---|
| On this iPhone | Apple Intelligence, on the iPhone itself | iOS 27, an iPhone that supports Apple Intelligence, with it switched on | Alerts, and events in Explore |
| On your server | Frigate’s GenAI, with the AI provider your server is set up for | Frigate’s genai settings on the server |
Alerts, Explore, and the AI summary in Review details |
On this iPhone Needs iOS 27
Section titled “On this iPhone ”Apple Intelligence describes the picture on this iPhone, and nothing is sent anywhere. It’s free, and it works with stock Frigate, no AI on the server needed.
Settings › Device Notification Settings › Picture Description has two switches:
- Describe alert pictures: adds a description to each alert, a few seconds after it appears. It needs Alert first, then add the picture on, under Media, so that writing a description never holds up an alert.
- Describe events in Explore: writes one when you open an event in Explore, under Apple Intelligence Description. It’s kept on this iPhone for 30 days. Describe again writes a fresh one.
The default alert wording shows the description until your server’s own arrives. If you’ve
written your own wording, add {{ios_ai}}: Add the description to my wording does it for
you. See Write your own alert wording.
Things to know:
- iOS decides when the on-device model may run, and sometimes declines in the background. When it does, that alert simply has no description. Settings › Notification Activity says when it happened.
- It’s generated text, and it can be wrong. It describes the one picture in the alert, so it can’t know what happened before or after.
- If the switches are greyed out, the footer under them says why: Apple Intelligence is off, the model is still downloading, the iPhone doesn’t support it, or it isn’t on iOS 27.
The description prompt
Section titled “The description prompt”Description prompt, under the two switches, changes what the model is asked. Most people never need it.
- Instructions: what to write and how, given to the model before every picture. Plain text only, because the alert’s details aren’t known yet when it’s read.
- Request: sent with the picture, and can use the alert’s details through
variables, such as
{{camera}}and{{zones}}. Sections work here too:{{#zones}}Zones: {{zones}}{{/zones}}. - Limits: Caption length, the longest description an alert carries (160 characters built in), and Reply length, where the model is stopped if it runs on (64 tokens built in).
- Try it: Test with the latest alert runs your newest alert through the same steps a real one takes, with your prompt, and shows the result as a notification.
Whatever you write, the reply is cut to one line and to the caption length. A model told
something is there tends to describe it even when it can’t see it, and what was seen and who was
recognised covers the whole event, not only the picture: keep that in mind before adding
{{subjects}} to the Request. Reset to the built-in wording and Use the built-in limits
undo your changes.
On your server
Section titled “On your server”If your Frigate server has GenAI set up, it writes its own descriptions with the AI provider you chose, and Fregata Mobile shows them:
- In alerts: the default wording adds your server’s description below the first line once
it’s written, replacing this iPhone’s. The alert updates in place, silently. Your own wording
can include it with
{{genai}}. - In Explore: under Server GenAI Description. Not generated yet means it hasn’t got to that event.
- In Review details: Frigate’s review summaries appear as the AI summary of the review that’s playing.
Frigate’s documentation, under Generative AI, explains how to set it up on your server.