================ ONE-SHOT BUILD CONTRACT (read first) ================
Build this in Google AI Studio "Build" in ONE shot — a complete, working app,
no follow-up turns. These are hard rules, not suggestions:
1. TARGET = Full-Stack Web (Node server runtime, secrets, Firebase allowed).
If you target Native Android instead, you MUST drop all server/DB/Workspace/
secrets and persist locally (Room / SharedPreferences) only.
2. PIN EVERY MODEL CALL — never let the agent auto-select (it downgrades on quota):
- Reasoning / text -> gemini-3.5-flash (thinkingLevel: minimal|low|medium|high)
- 4K image + legible text -> gemini-3-pro-image (image_size "4K", up to 14 refs)
- High-volume image -> gemini-3.1-flash-image
- Expressive TTS -> gemini-3.1-flash-tts-preview (inline tags e.g. [whispers])
- Realtime audio/video (WebSocket) -> gemini-3.1-flash-live-preview
- Sandboxed agent -> antigravity-preview-05-2026
3. DIVISION OF LABOR — the model ONLY parses/extracts to a strict responseSchema.
ALL math, money (store currency as integer minor units / cents), sorting,
balancing and graph logic run in deterministic TypeScript/Python. The model
must never compute totals, splits or balances itself.
4. responseSchema sanitation — no regex patterns, no fixed-length tuples, no
format validators in the schema (they crash the OpenAPI engine). Enforce those
in server-side code AFTER parsing the JSON.
5. responseSchema and google_search grounding are MUTUALLY EXCLUSIVE in one call.
6. CODEGEN — split large output into modular, single-responsibility files so no
file is truncated by the output-token cap.
7. Every external call gets a graceful fallback (e.g. manual paste if a Workspace
read fails). Never a silent dead end.
8. ROBUST STORAGE & CANVAS — Wrap all `localStorage`/`sessionStorage` operations (especially JSON parsing and writes) in `try-catch` blocks to prevent crashes in private windows or quota overflows. Canvas drawing elements must dynamically handle window resize and scale pixel density (`window.devicePixelRatio`) to avoid blurry graphics on retina displays.
=====================================================================
# MUST OBEY — Mobile-first build requirements
This app's PRIMARY surface is a mobile phone. Build it impeccably on mobile FIRST, then verify on tablet and desktop. Treat the rules below as non-negotiable hard constraints, not suggestions.
## Viewports to verify (every screen, every state)
- 320 px, 360 px, 375 px, 390 px, 414 px, 480 px
- 768 px, 834 px (iPad portrait / Pro 11)
- 1024 px, 1280 px, 1440 px, 1920 px, 2560 px
- Plus: 200% browser zoom, landscape orientation on every mobile width, iPhone with safe-area insets visible
## Hard layout rules
- Mobile-first CSS. Default styles target mobile; `@media (min-width: ...)` for larger viewports.
- Use `dvh` and `svh` instead of `vh` for full-height surfaces (iOS Safari URL-bar bug).
- Use `clamp()` for fluid typography across all viewports.
- Prefer container queries (`@container`) over media queries for component-level responsiveness.
- Use `min(100%, ...)` widths so content never overflows. Zero horizontal overflow at any viewport.
- Add `` to every page.
- Apply `padding: max(safe-area-inset-X, fallback)` on every edge-bleeding container so notched iPhones in landscape never clip content.
- Wide tables and code blocks scroll INSIDE their container (`overflow-x: auto`), never push the body.
- Use `background-attachment: scroll` on mobile, not `fixed` (iOS Safari repaint bug).
- Avoid `backdrop-filter` on animated elements. Use it sparingly on static surfaces only.
- **Canvas Scaling**: Canvases must dynamically scale with window resize events and properly handle high-DPI screens (`window.devicePixelRatio`). Set physical dimensions (`canvas.width`/`canvas.height`) using pixel ratio and render relative to this grid, using CSS to control responsive viewport scaling.
- **Robust Storage**: Every access to `localStorage`/`sessionStorage` (especially `JSON.parse` of loaded state or writes) MUST be wrapped in a `try-catch` block to handle disabled storage, private browsing mode, quota limits, or corrupted JSON gracefully. Fall back to a robust in-memory object store.
## Touch & accessibility
- Tap targets ≥ 44 × 44 px on touch (Apple HIG). Increase to 48 px under `@media (hover: none) and (pointer: coarse)`.
- All interactive controls reachable by keyboard with a visible focus ring; respect `:focus-visible`.
- Color contrast ≥ 4.5:1 for body text, 3:1 for UI components.
- All images have meaningful `alt`. Decorative images use `alt=""`.
- Respect `prefers-reduced-motion: reduce` — zero animation durations under that query.
- Forms validate inline; error messages are specific, not "Invalid input".
- Modals: focus trap, `Esc` closes, `role="dialog"`, `aria-modal="true"`, focus restored on close.
## Performance bar (Lighthouse mobile, throttled 3G/4G)
- LCP < 2.5 s · INP < 200 ms · CLS < 0.1
- JS bundle gzip < 200 KB mobile-first; lazy-load non-critical screens via `React.lazy` / dynamic imports.
- No render-blocking resources above the fold.
- Images: WebP/AVIF preferred, `loading="lazy"`, explicit `width`/`height` attributes (zero CLS), `srcset` for retina.
- Videos: `preload="metadata"`, low-resolution poster, max 720p mobile fallback. Never autoplay with audio.
- Fonts: `font-display: swap`; preload only the one used above the fold.
- Smooth scroll honoured via CSS `scroll-behavior: smooth` with reduced-motion fallback.
## Pre-ship mobile checklist (the deployer MUST verify before declaring done)
1. Open at 375 px in DevTools — every screen scrolls vertically only; zero horizontal scroll.
2. Browser zoom 200% — layout reflows without overlap.
3. iPhone Safari with the URL bar visible AND landscape — no content under the home indicator; no notch clipping.
4. iPad portrait (768 px) and landscape (1024 px) — no awkward gaps; tablet-specific breakpoints land cleanly.
5. Tap every interactive element with a thumb at real-device size — every target is easy to hit.
6. `prefers-reduced-motion: reduce` — every transition / animation skips cleanly, scroll-behavior becomes instant.
7. Lighthouse mobile score ≥ 90 across all 4 categories.
8. Zero `console.error` and zero CLS shift in real-device testing on a mid-tier Android (e.g. Pixel 6a) and an iPhone SE.
---
The original template starts below. All rules above apply on TOP of whatever this template specifies.
---
# Allergen Scanner
## 1. Project
**Allergen Scanner** is a one-tap "yes / no" reader for packaged-food
ingredient labels, built for the parent of a child with a severe food
allergy. The user points the phone at any packet on a supermarket shelf
— a snack bar, a tub of hummus, a bag of crisps, a chocolate, a school
lunchbox bento set, a takeaway-stamped sushi tray — and within about
three seconds gets a **big green check** ("safe for Aarav — no peanut,
no peanut-derived ingredient, no shared-facility warning") or a **big
red cross** ("not safe for Aarav — contains peanut" or "not safe for
Aarav — 'may contain peanut' shared-facility warning"). The verdict
points at the exact word or sentence on the packet that triggered it,
photographed and underlined.
This is the kind of app a dad builds — or finds in a gallery — when his
6-year-old has a severe peanut allergy and the supermarket has 47 new
biscuit SKUs since he last shopped, all with 5-point ingredient panels,
half of them in Vietnamese or Hebrew or Portuguese, half of them with
"may contain traces of peanut and tree nuts" buried in a paragraph two
font sizes smaller than the ingredient list. It is also the kind of app
an Israeli-American mum builds for her 4-year-old with a severe
cow's-milk-protein allergy who reacts to whey, casein, lactalbumin, and
the "produced in a facility that also processes milk" hummus tubs that
nobody else in the family thinks twice about — and the kind of app a
Vietnamese-Canadian dad builds for his 8-year-old with a life-threatening
shellfish allergy in a country where "natural flavour" can include shrimp
extract and where bánh phở packaging hides fish-sauce derivatives in a
single Vietnamese word he can't reliably scan in the freezer aisle.
Same shape of moment — a parent at a shelf with a phone and 90 seconds
before the kid melts down — different child, different allergen.
The single demo that proves the magic: hold the phone over **any** packet
of food. The camera fixes focus on the ingredient panel and the
"may-contain" line. In about three seconds the screen fills with one of
two states — a green check the height of half the screen, or a red cross
the same size — and underneath, in plain language: "Safe for Aarav. No
peanut. No 'may contain' warning. Last checked just now." Or: "Not safe
for Aarav. The ingredients list 'peanut oil' as the third ingredient."
The exact phrase is photographed, cropped, and underlined. Below the
verdict: a single button — "**Scan another packet**" — never "looks
fine, go ahead". The app never tells the parent the kid is safe; it
tells the parent what the packet says, with a verdict the parent can
trust because the app errs on the side of the kid.
And in the harder cases — packaging in scripts the parent doesn't read,
poor lighting, smudged labels, hand-written deli labels at the
butcher's counter, takeaway boxes with no ingredient panel at all, "may
contain" warnings printed two lines apart from the ingredient list, and
the ten thousand ways manufacturers hide allergen-derived ingredients
behind technical names (casein, whey, lactalbumin for milk; lecithin
sometimes from peanut; "natural flavour" that can legally include
crustacean extract in some jurisdictions; cochineal for some egg-allergic
kids; sodium caseinate; hydrolysed protein) — the app's first answer is
always the conservative one. When the image is unreadable, the verdict
is **red**. When the ingredient is ambiguous, the verdict is **red**.
When the parent's allergen list and the packet's wording cannot be
matched with high confidence, the verdict is **red** and the panel
explains exactly which word the app could not resolve. The model never
"probably safes" the kid.
**Tagline:** _Point the phone at any packet — in any language, on any
shelf — and know in 3 seconds whether your kid can eat it._
## 2. Target audience
- Parents of children with one or more severe IgE-mediated food
allergies (peanut, tree nuts, milk, egg, soy, wheat, sesame, fish,
crustaceans / molluscs)
- Parents of children with non-IgE allergies and conditions that still
require ingredient avoidance — FPIES (food protein-induced
enterocolitis), eosinophilic oesophagitis (EoE), severe atopic eczema
with confirmed food triggers
- Parents of children with coeliac disease who need strict gluten-free
with "may contain wheat" exclusion (this app is allergy-shaped but
the parsing covers gluten the same way)
- Adults living with their own severe food allergies — university
students newly shopping for themselves, adults diagnosed late in life
with sesame or shellfish allergy, pregnant adults with a new aversion
- Grandparents, aunts, uncles, babysitters, and the parents of a kid's
best friend — anyone who occasionally has the allergic child in their
care and needs to be able to scan independently
- Bilingual and multilingual families shopping across languages — the
Pakistani-British family at the Asian grocer, the Vietnamese-Canadian
family at the Korean supermarket, the Israeli-American family who
reads English and Hebrew labels in the same week
- Travelling families and immigrant families whose home labels are in
one language and whose supermarket labels are in another — Tagalog,
Vietnamese, Mandarin, Cantonese, Korean, Tamil, Hindi, Urdu, Bengali,
Punjabi, Amharic, Swahili, Farsi, Hebrew, Arabic, Polish, Portuguese
- School and nursery staff — the teaching assistant in the playground
with someone else's snack bar in their hand at 10:55am
- Allergist and dietician offices recommending a single tool to
newly-diagnosed families
## 3. Core value propositions
Surface these clearly through copy, visual emphasis, and section
ordering — they are the reasons users pick this app, and they encode
the safety stance.
- **One verdict per packet. Big enough to read at arm's length, with the
kid in a trolley seat tugging at your sleeve.** A check or a cross,
the size of half the screen. The verdict is for one named child at a
time — Aarav's check is not Noor's check, because Aarav is peanut
and Noor is milk. The parent picks who they're scanning for before
the camera opens.
- **Reads the actual packet, including the "may contain" line and the
facility warning.** Gemini 3.5 Flash's multimodal vision parses the
ingredient panel and the warning paragraph in one call, in any of
English, Vietnamese, Hebrew, Arabic, Mandarin, Cantonese, Korean,
Tamil, Hindi, Urdu, Bengali, Punjabi, Amharic, Swahili, Farsi,
Khmer, Tagalog, Polish, French, German, Spanish, Portuguese, Italian.
The full verbatim transcript is captured and stored — you can scroll
back to any past scan and see exactly what the packet said.
- **A curated cross-contact and synonym database for every allergen the
child has.** Casein, whey, lactalbumin, lactoglobulin, sodium
caseinate, ghee, lactose, condensed milk, milk solids, milk powder
for cow's-milk-protein. Arachis oil, ground nut, mandalona,
beer nuts for peanut. Cochineal / carmine / E120 surfaced when the
parent has set egg-related sensitivity. The database ships with the
app, is reviewed by allergy clinicians, and is the same on every
scan — not regenerated by the model each time.
- **The verdict comes with the receipts.** The crop of the packet where
the trigger word appears is shown, underlined, with the verbatim
source-language text and the English gloss. The parent can see *why*
the app said no. There is no black box.
- **When in doubt — RED. Always.** A hard rule built into the prompt,
the schema, and the UI. The model is forbidden from saying "probably
safe", "appears safe", "should be fine", "looks like it should be
okay". When the ingredient panel is unreadable, the verdict is red
with the reason "couldn't read the label clearly enough — try again,
brighter light, label flat". When the "may contain" line is missing
entirely, the verdict is red with the reason "no shared-facility
warning visible — check with staff or skip this packet".
- **The button at the bottom is always "Scan another packet" — never
"Looks fine, ship it".** A small detail that encodes the safety
stance and saves the parent's brain at the end of a long day.
- **Multi-child profiles in one household.** Aarav (peanut, tree nuts),
Noor (milk, egg), Mehmet (sesame). Each kid is a profile with their
own avatar; the home screen prompts "scanning for who?" the moment
the camera opens. A "scan for everyone" mode runs all profiles in
parallel and shows three verdicts at once.
- **History every scan. Repeat scans without retaking the photo.** Every
scan is saved with the verdict, the verbatim transcript, the cropped
trigger word, and the date. The same packet next week takes 0
seconds to re-verify — the parent sees the previous verdict the
moment they scan the front of pack. A weekly grocery becomes a
10-minute scan, not a 45-minute scan.
## 4. Features to build
- Camera-first home screen — the moment the app opens, the camera is
warm and pointed at the user's hands. No splash, no marketing.
- "Scanning for" picker — a row of round avatars across the top of the
camera viewfinder, one per child profile. Tap to switch profiles
between shots. The most-recently-used profile is selected by
default.
- Live viewfinder with ingredient-panel detection — the rectangle on
the screen finds the ingredient panel and the "may contain" warning
even when they are on different sides of the packet; an overlay
prompts the user to rotate the packet if the warning panel is on the
back
- Tap-to-scan capture — a single big shutter button, two-finger
brightness override, AE/AF locked on the panel
- Multi-photo capture for big packets — some packets carry the
ingredients on a wrap-around panel; a "next side" prompt lets the
user capture two or three sides as a single artefact
- Live verdict screen — a check or a cross filling the upper half of
the screen, the child's name and the allergen list immediately
underneath, the trigger text (if any) cropped from the packet and
shown verbatim with translation, and a single primary action: "Scan
another packet"
- Full verbatim transcript drawer — pulling up from the bottom of the
verdict screen reveals the entire ingredient panel as the model read
it, both source language and translation, with the trigger words
highlighted
- The "Why?" panel — every verdict, green or red, has a "Why this
verdict?" expander showing exactly which allergen-database synonyms
the model matched (and didn't match), and the confidence level for
each
- Curated allergen database — every common allergen and its synonyms,
derivatives, hidden names, and the legal-name variations across
EU / FDA / Codex Alimentarius / Israeli, Japanese, and Australian
labelling. Ships with the app; updated by us, not by the model
- Multi-profile per household — Aarav, Noor, Mehmet; each profile
carries one or more allergens, optional cross-contact-sensitivity
level (high → "may contain" warnings flip verdict to red; lower →
surface but don't flip), and optional notes (epinephrine prescribed
yes/no, last reaction date, school nurse contact)
- "Scan for everyone" mode — runs every child's profile in parallel
on a single photo and shows three small verdicts stacked instead of
one big one
- History list — every scan, sorted reverse-chronological, with packet
thumbnail, verdict, child name, date. Search by product name.
Re-scan jumps straight to the saved verdict and offers a re-photograph
in case the recipe has changed
- "I've seen this packet before" notice — when the front-of-pack
matches a recent scan for the same profile (within 60 days), the
prior verdict appears immediately and the parent can choose "use
prior verdict" or "re-scan, recipe may have changed"
- Pantry list — pin packets to a "safe at home" list and "never again"
list. Useful for the second parent at home who didn't do the
shopping
- Share with the babysitter — generate a one-page PDF of "what Aarav
can and cannot eat from our kitchen" with photos of the pinned
pantry. Magic-link share to a school nurse or grandparent.
- Allergist-shareable history — export a CSV of every scan in a date
range to email to the allergist before an appointment, useful when
hunting down a stealth exposure
- Settings for cross-contact stance per allergen per child —
"always red on 'may contain peanut'" vs "warn but don't flip" — set
by the parent under medical advice
- Calm-down UI — the verdict screen never plays sound; the haptic
pattern on red is a slow double-pulse, on green a single soft tap;
in shop-mode the screen brightens to be visible under fluorescent
light
- Offline-capable allergen database + degraded "couldn't reach the
model" mode — even with no signal, an OCR-and-database fallback can
catch the obvious cases; ambiguous cases are flagged "needs network
to be sure" and the verdict is **red**
- Accessibility — VoiceOver / TalkBack reads the verdict first, then
the trigger text. Haptics + a colour-blind-safe icon set
(check + cross with thick weights, not relying on green/red alone)
## 4b. Required Gemini capabilities + backend services
**This template's intelligence comes from the Gemini capabilities below.
Wire them up explicitly — don't substitute generic LLM calls. The
safety-relevant rule for every call is: when the model's answer is
uncertain, the verdict is red. This is encoded in the schema (the
verdict enum has no "probably safe" value), in the system instructions
(the model is forbidden from speculation), and in the UI (the parent
sees the reason for the red).**
### Gemini capabilities (the load-bearing intelligence)
- **Multimodal image input** (Gemini 3.5 Flash) — reads ingredient panels
in any of the languages and scripts listed in Section 3. Reads
packaging in poor lighting, glossy plastic with glare, curved
surfaces (cans, tubs), and faded or smudged labels. Reads
multipanel packets (where the ingredient list and the "may contain"
warning are on different sides) as a single multi-image call with
explicit "side 1 of 2 / side 2 of 2" headers.
- **Structured output / JSON Schema** — the response matches the
`LabelRead` schema below. Every field is typed; the schema is
included verbatim in the system instruction and as `responseSchema`.
The verdict enum is **`safe | unsafe_contains | unsafe_may_contain |
unsafe_unreadable | unsafe_ambiguous`** — there is no
"probably_safe", no "likely_safe", no "looks_fine".
- **Long context (1M tokens)** — used not on a single packet (which is
small) but on the household's scan history, the allergist export,
and the "find me a safe biscuit" follow-up flow. The verdict call
itself is a single image + a small JSON profile, well under any
context limit. The history / search call may include thousands of
prior scans; chunk by date or by product if the total exceeds 800k
tokens.
- **Search grounding** — used on the "is this brand reformulating?"
follow-up flow only, where the parent wants to check whether a
packet that was safe six months ago has changed recipe. Grounded
search hits manufacturer pages and the major allergy-recall RSS feeds
(US FDA, UK FSA, EFSA, Health Canada, Food Standards Australia New
Zealand). **Never used on the per-packet verdict call** — the
verdict is decided from the photographed packet, not from a web
search that might be wrong.
- **Gemini TTS** (`gemini-3.1-flash-tts-preview`) — narrates the
verdict aloud when accessibility settings request it. Locale matches
the parent's UI language, not the packet's language ("Not safe for
Aarav. The packet lists peanut oil.").
- **Nano Banana 2** (`gemini-3.1-flash-image`) — used **only** for the
household-pantry "safe at home / never again" printable; generates
the per-profile orientation card with the child's name in a friendly
hand. **Never used on the verdict screen itself** — the verdict
screen shows the real photograph the parent just took, not a
generated image.
- **Thinking levels** — `medium` for the primary read-and-decide call,
because allergen matching across synonyms in fifteen scripts is the
call's hard work. `low` for the "find me a safe biscuit" follow-up
search and the pantry-image generation prompt-building. **`medium`
is non-negotiable on the verdict call** — Flash dropped sesame
("ellu" in Tamil, "til" in Hindi, "shumshum" in Hebrew, "vunge" in
Tagalog) in early testing.
### Backend services
- **Auth — Required.** Firebase Auth with Google sign-in (auto-provisioned
by AI Studio Build). **Apple sign-in is optional but user-configured**:
it requires an Apple Developer account, Service ID, Key ID, and
private key wired into the Firebase Auth console. **Magic-link email**
(used for the babysitter / school nurse PDF share) requires the
sender domain to be authorised in Firebase Auth.
- **Database — Required.** Firestore for `households`, `children`,
`allergens` (the curated database is read-only system data, not
per-user), `scans`, `pantry_items`, `verdicts`, `audit_log`. The
audit log is non-negotiable: every verdict the app ever shows is
written immutably so that if a child reacts to a packet the app
marked safe, the family and the allergist can trace exactly what
the app said and why.
- **File storage — Required.** Firebase Storage for packet photographs
at upload resolution (kept forever for any scan the user has
pinned, otherwise pruned after 90 days; the user controls pruning).
**Storage is NOT auto-provisioned by AI Studio Build today** —
enable it in the Firebase console and wire the bucket name into the
AIS Build project before first scan.
- **Email — Required (transactional).** Babysitter / nurse PDF share
via Firebase Auth magic links; allergist CSV export via direct
email link; recall-alert notifications when a brand the household
has scanned is recalled by a national food-safety regulator.
- **Payments — Not needed for v1.** Free for personal use. A future
premium tier could offer multi-household ("scanning for the cousin
this weekend") and allergist-portal features. The verdict service
itself must always be free — gating safety behind a paywall is not
acceptable.
- **External APIs:** Gemini API for the intelligence; optional read of
the FDA Recall API, FSA Recall RSS, Health Canada Recall RSS, FSANZ
Recall RSS, EFSA RASFF alerts for the recall-alert feature. No
third-party allergen database — ours is curated.
**Environment variables:** every secret (Gemini API key, Firebase
service-account JSON, recall-RSS reader credentials if any) lives in
environment variables — never in client bundle. Include a `.env.example`.
**Auth + data privacy reminders:** never log secrets · never store
passwords in plain text · use HTTPS everywhere · honour 'delete my
account' inside the UI · explicit opt-in for any analytics · the
family's child-health data is sensitive (children's allergen profiles
are functionally medical data in most jurisdictions); store it in the
family's Firebase project with row-level security scoped to the
household. Use the Gemini API on the paid tier, where Google does not
use your content for model training, per the Gemini API Additional Terms.
The scanned packet photographs may include the child's name on a school
lunchbox or a delivery address on a takeaway box — never log full
images server-side outside of Firebase Storage; never include images in
analytics events.
**Read this first — prompt-craft rules that apply to every call in this template:**
1. **Name the model variant explicitly** in every Gemini API call. Do
2. **Grounded responses can wrap JSON in ```json fences or add prose preamble.** Server-side, strip fences and brace-extract:
```typescript
function safeExtractJSON(raw: string): T {
const clean = raw.replace(/```json\s*|```/gi, '').trim();
const s = clean.indexOf('{'); const e = clean.lastIndexOf('}');
if (s === -1 || e === -1) throw new Error('No JSON boundaries in grounded response');
return JSON.parse(clean.slice(s, e + 1)) as T;
}
```
3. **Strip unsupported Zod modifiers before passing to `responseSchema`** — Gemini's OpenAPI subset rejects `.regex()` / `pattern`, fixed-length `z.tuple()`, and other custom validators. Use a sanitizer that flattens tuples to length-2 arrays and removes regex patterns before serializing. Validate those constraints in middleware AFTER parsing.
not let the agent pick the model. See the per-call matrix below.
`gemini-3.5-flash` for the verdict call, full stop.
2. **Pin `thinkingLevel` explicitly** per call. See the matrix.
3. **Seed the JSON Schema as a fenced TypeScript / Zod block** in the
system instruction or `responseSchema` field. The literal schema is
below. **Convert the Zod schema to Gemini's `Schema` type via the
SDK helper** before passing to `responseSchema` — do NOT pass raw
Zod. **Numeric `min`/`max` constraints are documentation only inside
`responseSchema`; clamp on the server after the response arrives.**
4. **Pin the system instruction separately** from user input. Use the
`systemInstruction` field for persona + behavioural rules; use
`contents` for the photographed packet and the child profile JSON.
Never concatenate.
5. **Pre-declare tools as an enable/disable list** per call. The matrix
below names which tools are enabled per call. The verdict call has
no tools enabled — no web search, no function calls. The verdict is
decided from the photograph alone.
6. **State negative constraints explicitly** — they are listed below.
They are NOT "be careful" suggestions; they are hard rules the model
must follow.
### Per-call model + tools matrix
| Call | Model | thinkingLevel | Tools enabled |
|------|-------|---------------|---------------|
| Read packet + decide verdict → `LabelRead` schema | `gemini-3.5-flash` | medium | (none) |
| Re-translate transcript on demand (parent picks a different display language) | `gemini-3.5-flash` | low | (none) |
| "Find a safe alternative" follow-up — given a category + child profile, suggest brands | `gemini-3.5-flash` | low | `google_search` grounding (no `responseSchema` — see note) |
| Recall-alert check — given a list of brand+SKU strings and a date window, return recalls | `gemini-3.5-flash` | low | `google_search` grounding |
| Generate pantry-orientation card (Aarav's safe-at-home printable) | `gemini-3.1-flash-image` | n/a | n/a |
| Read verdict aloud (accessibility) | `gemini-3.1-flash-tts-preview` | n/a | n/a |
*Note for builders:* on TTS and image-generation calls, omit
`thinkingConfig` entirely — the field is not supported on those models.
The `n/a` cells in this matrix are documentation only; do not serialise
them into the request body. On the two grounded-search calls,
`responseSchema` and `google_search` cannot be combined in the same
Gemini call today — instruct the model to emit JSON in the text body
and parse server-side; read citation URLs from
`response.groundingMetadata.groundingChunks[].web.uri`.
### Primary structured-output schema (seed this verbatim in the prompt)
```typescript
import { z } from "zod";
const AllergenMatch = z.object({
allergen_id: z.string(), // "peanut", "milk", "tree_nut.cashew"
matched_term_verbatim: z.string(), // "peanut oil" — as written on packet
matched_term_source_language: z.string(), // BCP-47, "en", "vi", "he"
matched_term_translation: z.string(), // "peanut oil" — English gloss for UI
match_kind: z.enum([
"direct_ingredient", // "peanut" appears in ingredient list
"synonym", // "groundnut", "arachis", "mandalona"
"derivative", // "peanut oil", "casein", "whey"
"may_contain_warning", // "may contain peanut"
"shared_facility_warning", // "produced in a facility that processes nuts"
"ambiguous_natural_flavour", // "natural flavour" without source spec
"ambiguous_other", // anything the model is not sure about
]),
match_confidence: z.number().min(0).max(1),
position_on_packet: z.string(), // "ingredient line 3", "may-contain warning, line 2"
crop_coordinates: z.object({ // bounding box, 0-1 normalised
x: z.number().min(0).max(1),
y: z.number().min(0).max(1),
width: z.number().min(0).max(1),
height: z.number().min(0).max(1),
page_index: z.number().int().min(0), // for multi-side packets
}),
});
const TranscriptBlock = z.object({
block_label: z.enum([
"ingredient_list",
"may_contain_warning",
"shared_facility_warning",
"nutrition_panel", // captured but not parsed for allergens
"product_name",
"front_of_pack",
"barcode_region",
"other_visible_text",
]),
text_verbatim: z.string(), // exactly as written
text_translation_en: z.string().nullable(), // English gloss for UI
source_language: z.string(), // BCP-47
page_index: z.number().int().min(0),
});
const LabelRead = z.object({
scan_id: z.string(),
child_profile_id: z.string(), // who this verdict is for
child_name: z.string(), // copied in for audit
child_allergens: z.array(z.string()), // copied in for audit
cross_contact_stance: z.enum([
"always_flip_red_on_may_contain",
"always_flip_red_on_shared_facility",
"warn_but_dont_flip", // very rare, only on medical advice
]),
packet_image_uris: z.array(z.string()), // one or more sides
product_name_verbatim: z.string().nullable(),
product_name_translation_en: z.string().nullable(),
brand_verbatim: z.string().nullable(),
barcode_verbatim: z.string().nullable(), // if visible, for history matching
source_languages_detected: z.array(z.string()), // BCP-47 list
transcript_blocks: z.array(TranscriptBlock),
ingredient_panel_found: z.boolean(), // false → verdict will be unsafe_unreadable
may_contain_warning_found: z.boolean(), // false → verdict policy fires
shared_facility_warning_found: z.boolean(),
matched_allergens: z.array(AllergenMatch), // any allergen the model identified
verdict: z.enum([
"safe",
"unsafe_contains", // direct ingredient or derivative
"unsafe_may_contain", // may-contain or shared-facility warning
"unsafe_unreadable", // photo too poor to be sure
"unsafe_ambiguous", // "natural flavour" or similar
]),
verdict_reason_short: z.string(), // ≤ 12 words, shown under the big icon
verdict_reason_long: z.string(), // 2-4 sentences, shown in the "Why?" panel
trigger_block_index: z.number().int().nullable(), // index into transcript_blocks for the receipt
trigger_term: z.string().nullable(), // the specific word, verbatim
unreadable_reasons: z.array(z.enum([
"blur", "glare", "low_light", "occluded",
"label_too_small", "language_not_supported",
"missing_panel", "torn_or_damaged", "other",
])),
recommended_next_action: z.enum([
"scan_another_packet",
"retry_with_better_photo",
"ask_staff_to_check_facility",
"skip_this_packet",
"consult_a_human", // for any "I'm not sure" path
]),
reading_confidence: z.number().min(0).max(1),
flagged_for_user_review: z.array(z.object({
field_path: z.string(),
reason: z.string(),
})),
});
type LabelRead = z.infer;
```
### Common failure modes (and how to avoid them)
- **Agent picks `gemini-3.5-flash` for the verdict call to save quota.**
Pin `gemini-3.5-flash` explicitly. Flash dropped sesame variants ("ellu",
"til", "shumshum", "vunge") in early testing, smoothed over
"may contain" warnings, and missed Vietnamese "đậu phộng" (peanut)
when the panel was at an angle. Verdict call is `gemini-3.5-flash`, full
stop.
- **Model returns "probably safe" or "appears safe".** The schema has
no such verdict. Validate the response server-side — if `verdict`
is not one of the five enum values, retry once, then fall back to
`unsafe_ambiguous` with a server-generated reason "model returned
an unexpected verdict; treating as not safe".
- **Model summarises the ingredient list instead of transcribing it.**
The system instruction pins "verbatim text only in transcript_blocks
— no summarisation, no normalisation, no expansion of abbreviations".
Unit-test against a packet with "skd milk" and verify the transcript
block contains "skd milk" not "skimmed milk".
- **Model translates ingredient terms into the parent's UI language
before allergen-matching.** The matching happens against the
curated database in the original language and in English; the
translation is for the parent's display, not for the matching. Pin
this in the system instruction.
- **"May contain" warning treated as decorative.** Many parents'
worst day comes from a "may contain traces of peanut" line two
lines below the ingredient panel. The system instruction explicitly
treats may-contain and shared-facility lines as first-class signals
— `may_contain_warning_found = true` is a required field, and a
matched allergen with `match_kind = may_contain_warning` will flip
the verdict to `unsafe_may_contain` whenever the child's
`cross_contact_stance` says so.
- **Photo at an angle reads "peanut" as "pea unt" or splits "milk" into
two tokens across a fold.** The model should still match the
database via the synonym table (which includes common OCR-error
forms). When confidence on a partial token is below 0.85, the
verdict is `unsafe_ambiguous`, not `safe`.
- **Model invents a "may contain" warning that isn't on the packet.**
Hard rule in the system instruction: `may_contain_warning_found`
must be set to true *only* if a verbatim warning text is captured
in `transcript_blocks`. The verdict reason quotes that text.
- **Model confuses the front-of-pack "no nuts!" marketing claim with a
binding allergen statement.** "Nut-free" on the front-of-pack does
not override "may contain peanut" on the ingredient panel. The
ingredient panel is authoritative. The marketing front-of-pack is
captured as `front_of_pack` but does not feed the verdict.
- **Model says safe when no ingredient panel was visible.** This is
the most dangerous failure mode. Hard rule: if
`ingredient_panel_found = false`, the verdict is **always**
`unsafe_unreadable`, regardless of whatever else the model thinks
it saw.
- **Multi-side packet submitted as separate calls, losing the link
between the ingredients (side A) and the "may contain" warning
(side B).** Submit as one multi-image call with explicit "side 1
of 2 / side 2 of 2" labels.
### Negative constraints (hard rules)
- Do NOT output any verdict outside the five-value enum. There is no
"probably safe", no "likely safe", no "appears safe", no "should
be fine", no "looks okay". The schema enforces this; the system
instruction reinforces it; the server validates it.
- Do NOT mark a packet `safe` when the photograph quality, partial
occlusion, missing panel, or unsupported language means the model
cannot resolve every ingredient. Use `unsafe_unreadable` and explain
why.
- Do NOT mark a packet `safe` when "natural flavour", "spices", or any
unspecified ingredient appears that could legally contain the
child's allergen in the packet's jurisdiction. Use
`unsafe_ambiguous` and explain.
- Do NOT extrapolate beyond the photograph. If a regional version of
a brand uses different ingredients elsewhere, the model has no way
to know that from this photograph — it judges this packet, in this
shop, on this day.
- Do NOT translate or normalise the verbatim text. Ingredient panels
are captured exactly as written, including misspellings,
abbreviations, and regional spellings. Translation is a separate
field for the user's display.
- Do NOT use marketing front-of-pack claims as evidence in the
verdict. "Nut-free", "gluten-free", "suitable for allergy
sufferers" on the front of pack are captured but do not flip the
verdict to safe; the ingredient panel is authoritative.
- Do NOT include any commentary, qualifier, or hedge outside the
structured output. No "however, you may want to check with…", no
"if you're unsure…". The structured fields carry the message.
- Do NOT log the photographed packet image to analytics or to the
model's training pipeline. Use the Gemini API on the paid tier,
where Google does not use your content for model training, per the
Gemini API Additional Terms.
- Do NOT diagnose the child or recommend medical action. The verdict
is about the packet, not about the child. Anaphylaxis treatment is
decided by the family's emergency action plan and their
prescriber, not by this app.
- Do NOT auto-update a previously-given verdict if the recipe has
changed on a later scan. Each scan is its own verdict, immutable.
If the recall-alert flow finds a relevant change, raise a
notification but never silently rewrite history.
- Do NOT permit a "verdict override" by the parent that flips an
unsafe verdict to safe. The parent can override safe → unsafe (for
their own reasons — a kid had a reaction to a brand the app said
was clean) but never unsafe → safe.
### Per-call `systemInstruction` strings
Use these as the literal `systemInstruction` field for each Gemini API
call the built app makes. They complement the series-wide rules
already uploaded as the global instructions file
(`00-series-instructions.txt`).
### Call: Read packet + decide verdict → `LabelRead` schema
Model: `gemini-3.5-flash` · thinkingLevel: medium · Tools: (none)
```
You are reading a photograph of a packaged-food product to decide
whether a named child can eat it. The child's name and the child's
allergens are provided in the user input alongside the photograph.
The stakes are not theoretical. If your verdict is wrong and you say
"safe" on a packet that contains the child's allergen, the child can
have anaphylaxis. Treat every verdict as if a paediatric allergist will
read your reasoning tomorrow morning with the child's medical record
open.
You may receive one image (one side of the packet) or several images
(multiple sides of the same packet, in order). Multi-side packets are
submitted as a single call with explicit "side 1 of N / side 2 of N"
headers. Upload via the Gemini Files API (`files/*` resource name) or send as
`inlineData` (base64). Do NOT expect Firebase Storage public URLs to be
fetched server-side.
Languages and scripts you may encounter on packets include English,
Vietnamese (chữ Quốc ngữ), Hebrew, Arabic, Mandarin and Cantonese in
traditional or simplified Chinese characters, Korean (Hangul), Tamil,
Hindi in Devanagari, Urdu in Nastaliq, Bengali, Punjabi in Gurmukhi or
Shahmukhi, Amharic in Ge'ez script, Swahili, Farsi in Nastaliq, Khmer,
Tagalog (Filipino), Polish, French, German, Spanish, Portuguese,
Italian, Russian, Greek, Turkish, Indonesian, Malay, Thai, and
Japanese (kanji + kana).
For each photograph, identify and transcribe verbatim:
- the product name and brand (front-of-pack)
- the ingredient panel (the authoritative list of ingredients)
- any "may contain" warning (advisory cross-contact statement)
- any "produced in a facility that also processes" warning (shared-
facility statement)
- the barcode region (verbatim digits if visible)
- any other visible text that may be relevant
The transcript is verbatim. Do NOT summarise. Do NOT expand
abbreviations. Do NOT correct misspellings. "skd milk" stays "skd
milk" — the translation field is where the explanation goes.
For every allergen on the child's profile, search the transcript blocks
for any direct match, synonym, derivative, may-contain warning, or
shared-facility warning. The curated synonym database for each allergen
is provided in the user input as `allergen_database`. Use it as the
authoritative source for what counts as a match.
Decide a single `verdict` from the closed enum:
- `safe` — every ingredient is identifiable, no match to any of the
child's allergens, no may-contain warning, no shared-facility warning
(or the child's cross_contact_stance is set to "warn_but_dont_flip"
for the warning category found).
- `unsafe_contains` — a direct ingredient or derivative match was
found in the ingredient panel.
- `unsafe_may_contain` — a may-contain or shared-facility warning was
found and the child's cross_contact_stance flips the verdict.
- `unsafe_unreadable` — the photograph quality, missing panel,
unsupported language, or partial occlusion means you cannot resolve
every ingredient with high confidence.
- `unsafe_ambiguous` — "natural flavour", "spices", "flavouring",
"seasoning", or any unspecified ingredient appears that could legally
contain the child's allergen.
Hard rules — the safety stance:
- There is no "probably safe" verdict. There is no "appears safe"
verdict. There is no "should be fine" verdict. If you cannot
confidently say `safe`, you say one of the four unsafe values.
- If `ingredient_panel_found` is false, the verdict is ALWAYS
`unsafe_unreadable`, regardless of any other text you read on the
packet.
- If a may-contain warning is present and the child's
cross_contact_stance is "always_flip_red_on_may_contain", the
verdict is `unsafe_may_contain`. Do not second-guess the stance.
- The front-of-pack marketing claim ("nut-free", "allergen-friendly",
"kids' snack") is NOT authoritative. The ingredient panel is
authoritative. Marketing claims are captured as `front_of_pack`
blocks but do not affect the verdict.
- If you matched any allergen, populate `trigger_block_index` and
`trigger_term` so the UI can show the parent the exact word.
- `verdict_reason_short` is ≤ 12 words, plain language, names the
child. Example: "Not safe for Aarav. Contains peanut oil."
- `verdict_reason_long` is 2–4 sentences. It explains the match, the
position on the packet, and (for unsafe verdicts) the
recommended_next_action.
- Populate `unreadable_reasons` only when the verdict is
`unsafe_unreadable`. For other verdicts, leave it empty.
- Do NOT extrapolate to recall information, to other countries'
versions of this product, or to the manufacturer's other products.
You judge this packet, in this photograph, on this day.
Output ONLY the LabelRead JSON matching the provided schema.
No commentary. JSON only.
```
---
### Call: Re-translate transcript on demand
Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: (none)
```
You receive a transcript captured from a food packet (one or more
`TranscriptBlock` records, with `text_verbatim` and
`source_language`) and a target display language. Re-render
`text_translation_en` (or to whichever language the parent has
selected) for each block.
Hard rules:
- Translate the meaning, not the marketing. Render "yummy", "tasty",
"for the whole family" as their plain equivalents — never embellish.
- Preserve the structure of the ingredient list. If the source uses
parenthetical sub-ingredients ("chocolate (cocoa mass, sugar, milk
fat, soya lecithin, vanillin)"), keep the parentheses and the order.
- Preserve commas and semicolons exactly. Many ingredient lists use
punctuation as the legal boundary between ingredients.
- Do NOT change the order of ingredients. Most jurisdictions require
ingredients in descending order of weight; the order is information.
- For may-contain and shared-facility warnings, translate the exact
legal phrase: "may contain peanuts" — not "could possibly contain
peanuts". The legal weight differs.
- Do not add ingredients that are not in the source. Do not omit
ingredients in the source. The transcript is the source of truth.
Output: an array of `TranscriptBlock` records with the translation
field updated. No commentary outside the structured output.
```
---
### Call: Find a safe alternative
Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: `google_search` grounding
```
You receive a product category (e.g. "chocolate biscuits", "instant
ramen", "fish fingers") and a child's allergen profile (e.g. peanut,
tree nuts, no may-contain). Your task: suggest 3–5 packaged brands
the parent could investigate that are commonly available in the
parent's country and that publicly state allergen-friendly
formulations for the child's allergen set.
Use `google_search` grounding to anchor every suggestion to the
brand's current allergen statement. Cite the source URL for each
brand.
Hard rules:
- This is a suggestion to investigate at the shelf, NOT a verdict.
The parent will scan the packet with this app to decide.
- Do not say "safe". Say "publicly states peanut-free formulation,
verify the specific SKU at the shelf with the scanner".
- If the brand is regional, say which country / region. Do not
recommend a brand that does not sell in the parent's country.
- Prefer brands that explicitly state their cross-contact policy
on their own website over brands that do not.
- If no public statement is available, do not include the brand.
- Do not include manufacturer-PR copy verbatim. Summarise in plain
language and cite the URL.
Output the suggestions as JSON in the text body (NOT via
`responseSchema` — `responseSchema` and `google_search` cannot be
combined in the same Gemini call today). Server-side: parse the JSON,
read citation URLs from
`response.groundingMetadata.groundingChunks[].web.uri` — do not
ask the model to put URLs in the JSON body, it will hallucinate them.
No commentary outside the JSON.
```
---
### Call: Recall-alert check
Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: `google_search` grounding
```
You receive a list of (brand, product_name, country) tuples from the
household's scan history within the last 60 days, and a date window.
Your task: identify any of those products that have been the subject
of a national-regulator allergen recall in the date window.
Sources to ground: FDA recall feed, FSA UK recalls, Health Canada
recalls, FSANZ recalls, EFSA RASFF alerts, the regulator equivalent
for the parent's country.
Hard rules:
- Only report a recall if you can cite a regulator URL. Manufacturer
press releases without a regulator citation do not count.
- A recall counts as relevant only if the recall reason names an
allergen (e.g. "undeclared peanut", "undeclared milk", "may contain
undeclared sesame").
- Do NOT extrapolate from a recall in one country to the same brand
in another country.
- Do NOT speculate about future recalls. Report only what has happened.
Output the matches as JSON in the text body (not via `responseSchema`).
Read citation URLs server-side from
`response.groundingMetadata.groundingChunks[].web.uri`.
No commentary outside the JSON.
```
---
### Call: Generate pantry-orientation card
Model: `gemini-3.1-flash-image` · n/a · n/a
```
Generate a single warm, friendly illustration suitable for a printed
A5 pantry card to be stuck on the fridge of a family member or
babysitter. The card is for the child named in the prompt, with their
allergen set.
Style: hand-illustrated, warm pastels, gentle line, a child's-drawing
warmth without being childish. The child's name is rendered in
hand-lettered display type at the top. The allergens are listed under
a small heading "Things to avoid for [name]" with a simple icon for
each (peanut, tree nut, milk, egg, soy, wheat, sesame, fish,
crustacean). One short sentence at the bottom in friendly hand
lettering: "When in doubt, check the packet with the scanner — or
ask [name]'s parent." No photographs of the child. No medical
iconography.
Do not include the words "safe" or "allergen" in large type. The
card's job is orientation, not decision-making — the scanner makes
decisions.
Output a single PNG, 1748 × 2480 px (A5 at 300 dpi).
```
---
### Call: Read verdict aloud (accessibility)
Model: `gemini-3.1-flash-tts-preview` · n/a · n/a
```
Voice: calm, clear, unhurried. Pick the Gemini 2.5 Flash TTS voice
whose `languageCode` matches the parent's UI language (not the
packet's language). Pronunciation will follow that locale
automatically.
Pre-process the text before sending it to TTS:
- Read `verdict_reason_short` first. Pause with an ellipsis (`…`).
Then read `verdict_reason_long`.
- Pronounce the child's name and the allergen name slowly. If the
parent's profile includes a phonetic spelling of the child's name
("AAH-rahv" for Aarav), prepend it inline so the voice reads the
intended pronunciation. Gemini 2.5 TTS does not expose SSML
`` — these inline cues are how you signal pronunciation.
- At sentence boundaries, blank-line plus em-dash (`—`) signals a
longer pause. Gemini 2.5 TTS does not support `` — use
the textual cues.
- Target rate: ~140 words per minute on safe verdicts; ~120 wpm on
unsafe verdicts. The slower pace on red verdicts is intentional —
the parent needs to hear the trigger word clearly.
Style direction: prepend ONE short directive sentence to the text
input, exactly like: "Read calmly and clearly, as a friend gently
telling a parent what's on the packet. …". There is no separate
`style` API field on Gemini 2.5 TTS; the directive sentence inside
the input is how style is conveyed.
Phoneme overrides for child names in non-English-speaking locales
are NOT exposed by Gemini 2.5 TTS — no SSML `` tag.
Pronunciation comes from the chosen voice's native locale and from
the inline phonetic cue.
Mid-call voice switching is not supported. The verdict is read in
one voice — the parent's UI-language voice, never the packet's
language voice. (Reading "đậu phộng" aloud in a Vietnamese voice and
then continuing in English is not supported; render in the parent's
language and treat the source-language word as a quoted term.)
```
## 5. Use cases & content to include
Build dedicated UI sections or flows for each of these — they tell you
what content the app must support.
- **The supermarket aisle.** Aarav's dad is at the biscuit aisle with
a trolley and a 6-year-old in the seat. He picks up a new brand of
chocolate chip cookies. He taps the home screen, the camera opens,
he holds the phone over the ingredient panel, and three seconds
later sees a big red cross — "Not safe for Aarav. The packet says
'may contain traces of peanut'." He puts it back, picks the next
one, scans, sees a green check. Total time: 15 seconds for two
packets.
- **The Israeli-American household.** Noor's mum is at the kosher
hummus shelf. She scans a tub labelled in Hebrew. The app reads
"תכולה: גרגירי חומוס, טחינה, מיץ לימון" and the warning "ייתכן
שמכיל חלב" — "may contain milk" — and shows a big red cross. The
trigger phrase is photographed, underlined, and shown in Hebrew
with the English gloss "may contain milk" underneath.
- **The Vietnamese-Canadian family at the Korean supermarket.** Linh's
dad is buying instant ramen for his 8-year-old daughter with a
severe shellfish allergy. The packet's ingredient panel is in
Korean and Vietnamese. The app catches "건새우 분말" (dried
shrimp powder) on the Korean side and shows a red cross. The
English gloss "dried shrimp powder" appears under the photographed
trigger phrase.
- **The babysitter's Saturday.** Aarav's grandparents are watching
him. They open the household pantry list on the fridge tablet and
see green-checked photographs of the snacks they're cleared to
give. When grandma wants to open a new bag of crisps that isn't on
the list, she scans it from the kitchen — same camera, same
verdict, same one-tap.
- **The school holiday in Lisbon.** The family travel; Aarav's dad
scans Portuguese-labelled packets in the Pingo Doce. The app reads
Portuguese ingredient panels and "pode conter amendoim" warnings.
The verdict reasoning still uses Aarav's name in English; the
trigger phrase shows in Portuguese with the English gloss.
- **The takeaway with no ingredient panel.** Aarav's dad orders a
bowl of Thai noodles at a market stall and tries to scan the
takeaway box. There is no ingredient panel. The verdict is
`unsafe_unreadable` with the reason "no ingredient panel visible —
ask the cook directly about peanut" and the recommended action
"ask_staff_to_check_facility". The button at the bottom reads
"Skip this one".
- **The recipe has changed.** Three months ago Aarav's dad scanned
the same brand of oat bar and saw a green check. Today he scans
again and the app catches that the brand has reformulated to
include "may contain peanut". The history view shows the prior
green verdict (immutable, with date) and the new red verdict (also
immutable, with date). The household alert email goes out to the
second parent.
- **The recall.** Two weeks after the family scanned a brand of
chocolate biscuits and got a green check, FSA UK issues a recall
citing undeclared peanut in a batch. The recall-alert flow fires
a notification: "A brand you've scanned has been recalled. See
details." The history view shows the original safe scan with a
red overlay and the citation URL of the recall.
- **The newly diagnosed family.** A six-year-old has just been
diagnosed with a peanut and tree-nut allergy after a hospital
admission. The dad sets up the app on the cab ride home from the
hospital — child name, allergens, cross-contact stance
(always-flip-red on may-contain, per the allergist's standing
advice). His first scan that night is the bag of trail mix in the
cupboard the kid was eating the day before. Red cross.
- **The auntie in the kitchen.** The family's auntie is visiting and
is on the household's "trusted carers" list. She opens the same
app on her own phone, signed into the same household, and the
pantry list and history are there. She scans the new yogurt she
brought without needing to call the kid's parents.
## 6. Page structure
Build the following screens / sections in this order. Adjust copy to
fit the voice, but keep the structural intent.
1. **Welcome / sign-in.** A photographed-looking shot of a dad's hand
holding a phone over a chocolate biscuit packet on a supermarket
shelf, the screen of the phone showing a big green check. One
paragraph: "Point the phone at any packet — in any language — and
know in 3 seconds whether your kid can eat it." Single Google
sign-in button; Apple sign-in next to it. Below: "Try the demo
household" → loads the demo profiles from section 8a.
2. **First-run profile setup.** Three short forms: household name,
first child profile (name, allergens — multi-select from a
clinician-reviewed list, cross-contact stance per allergen),
optional phonetic spelling for the TTS readout. "Add another
child" button at the end.
3. **Home screen — camera-first.** The viewfinder is the home screen.
No splash, no marketing. Across the top: a row of round avatars,
one per child profile, with the most-recently-used one selected.
Above the shutter button: "Scanning for **Aarav** (peanut, tree
nuts)" in plain language. Below the shutter button: a small
history tile showing the last scan's verdict thumbnail.
4. **Camera viewfinder + capture.** Live frame detection draws a thin
rectangle around the ingredient panel and a second rectangle
around any may-contain warning. A small "more sides?" affordance
lets the user capture two or three sides of a wrap-around packet
as one artefact. AE/AF lock on tap; brightness override with
two-finger drag.
5. **Verdict screen.** The big icon — green check or red cross — fills
the upper half of the screen. The child's name and the allergen
list are immediately under the icon ("Not safe for Aarav. Peanut
detected."). The cropped trigger word from the packet is shown
centred, underlined, with the source-language text on top and the
English gloss underneath. A single primary action button at the
bottom: "**Scan another packet**". A secondary "Why?" expander
pulls up from the bottom and reveals the full verdict reasoning
and the verbatim transcript blocks.
6. **History list.** Reverse-chronological. Each entry: packet
thumbnail, product name, brand, child name, verdict icon, scan
date. Pulls down to search by product name. Tap an entry to see
the full saved verdict (immutable). A re-scan button on each entry
takes the user to the camera with the prior scan held in memory
for comparison.
7. **Pantry list.** Two columns: "Safe at home for **Aarav**" and
"Never again for **Aarav**". Each item is a card with the
photographed packet thumbnail, the brand, the date last verified,
and a small badge if the recipe has changed since (the recall-
alert badge sits here too). Each child has their own pantry view;
the household view shows the intersection.
8. **Profiles & settings.** Per child: name, phonetic spelling, avatar,
allergens, cross-contact stance per allergen, prescribed
epinephrine yes/no (no medical advice from the app — this is for
the share-with-babysitter PDF), allergist contact (for share),
school nurse contact (for share), the date the child was
diagnosed (for the timeline view in the allergist export).
9. **Share with the babysitter.** A modal: "Share Aarav's pantry list
with someone for tonight". Two options: magic-link email to a
one-page web view (live, reflects the household pantry in
real-time); or generate a printable PDF (static, reflects the
pantry at the moment of generation). The web view is read-only;
the babysitter cannot mark items.
10. **Allergist export.** Date range picker → CSV of every scan
(date, brand, product, verdict, trigger term, child, language).
Emailed via Firebase Auth magic link to the allergist's address
the parent enters.
11. **Recall alerts.** A list of recalls flagged against the
household's recent scans. Each card shows the original scan, the
recall date and source, the regulator citation URL, and a
"remove from pantry" action.
12. **Footer.** "Made for the parent at the shelf at 5pm with the
trolley and the meltdown." Privacy: "Your child's profile is
yours. We never train on it." Capabilities `(i)` icon in header.
## 6b. First-visit onboarding
Show a **first-visit onboarding** the first time a visitor lands on the
app (detect via `localStorage` flag; do not show on return visits).
Three slides, dismissible at any time. Persistent re-entry: a `?` icon
in the header reopens it.
**Slide 1 — What this is.**
- Headline: "Welcome to Allergen Scanner."
- Subhead: "Point the phone at any packet — in any language — and know
in 3 seconds whether your kid can eat it."
- One paragraph (≤ 60 words) explaining who this is for and what makes
it different from a generic OCR app: the verdict is per-child, the
may-contain warning is treated as first-class, the front-of-pack
marketing claim is ignored, the verdict is always conservative —
when in doubt, **red**.
- Visual: a small annotated illustration of a chocolate biscuit packet
with the relevant zones labelled — ingredient panel, may-contain
warning, shared-facility statement, front-of-pack claim (with a
small note "this part is NOT what the app uses to decide") — not
a generic packet icon.
**Slide 2 — Try it now.**
- One short prompt: "Try with the demo household".
- A live demo input pre-loaded with three child profiles and three
sample packets from section 8a. The parent sees one green check
and two red crosses, and can tap into each to see the reasoning.
- 1-2 sentences pointing at *the specific page elements* where the
Gemini magic happens (the Hebrew "may contain milk" line on the
hummus tub, the Vietnamese "đậu phộng" on the satay sauce, the
curated synonym match on "casein" in the protein bar).
**Slide 3 — How to remix this.**
- Headline: "Make this yours."
- Three short bullets:
- "Edit the curated allergen database in
`/data/allergens/` to add allergens your family lives with."
- "Adjust the prompts in `/server/prompts/` to tune the
safety stance for your family's medical advice."
- "Wire up your Gemini API key and Firebase project via the env-var
list in the capabilities panel."
- Primary CTA: "Use this template" → links to AI Studio Build remix
entry point.
- Secondary: "Just exploring — close" (sets localStorage flag, never
auto-shows again).
**Accessibility:** focus trap, `Esc` closes, `role="dialog"`,
`aria-modal="true"`, `aria-labelledby`, focus restored to trigger on
close. Respect `prefers-reduced-motion`.
**Don't:**
- Don't gate the camera behind the modal. The page beneath must be
fully usable — the family at the supermarket may not have time for
a tour.
- Don't auto-reshow on return visits. Use
`localStorage['onboarding-seen-v1']`.
- Don't include unrelated CTAs (newsletter signup, social follow).
Keep it about the template only.
## 6c. Capabilities info button (persistent in header)
Add a persistent `(i)` icon in the top-right of the header (next to
the primary nav). Click → opens a modal/panel titled **"What powers
this app"**.
**Panel contents (in this order):**
**Gemini capabilities used (the hero list):**
- **Gemini 3.5 Flash (multimodal)** — reads the ingredient panel and the
may-contain warning in any of the languages and scripts the app
supports. One call per packet (one or more sides), structured
output, `thinkingLevel: medium`. The verdict is decided in this
call; no web search is used.
- **Gemini 3.5 Flash (multilingual)** — translates the verbatim transcript
into the parent's UI language for display, without altering the
matching that already happened against the curated allergen
database in the source language.
- **Gemini 3.5 Flash (long context)** — used on the history search ("show
me every red verdict in the last 90 days for Aarav") and on the
allergist export. Not used on the per-packet verdict — the verdict
is a single image and a small profile JSON, well within ordinary
context.
- **Gemini 3.5 Flash + grounded search** — used on two follow-up flows
only: "find me a safe biscuit brand to investigate" and the
recall-alert check. Never on the verdict call itself.
- **Gemini TTS** — reads the verdict aloud when accessibility is
enabled, in the parent's UI language, with the trigger word
pronounced slowly.
- **Nano Banana 2 (Gemini 3.5 Flash Image)** — generates the printable
pantry-orientation card for the babysitter. Never used on the
verdict screen — the verdict shows the real packet photograph the
parent just took.
- **Firebase Auth** — Google and Apple sign-in, family invitations via
magic links.
- **Firestore** — stores your household, child profiles, scans, and
pantry. Real-time sync across the household's devices so both
parents see the same pantry list.
- **Firebase Storage** — keeps the packet photographs at upload
resolution for pinned items, prunes others after 90 days.
- **Cost note** — see the detailed breakdown in 6d. A typical
household scanning 200 packets a month costs about $2.40 of Gemini
API spend, total.
- **Privacy note** — your child's allergen profile is sensitive. This
app uses the Gemini API on the paid tier, where Google does not
use your content for model training, per the Gemini API Additional
Terms. The packet photographs may show your delivery address on a
takeaway sticker — we never log full images in analytics.
**The safety stance, in plain English:**
- The verdict is always one of five: safe, not safe (contains), not
safe (may contain), not safe (couldn't read clearly), not safe
(ambiguous ingredient).
- There is no "probably safe". When in doubt, the verdict is **red**.
- The front-of-pack marketing claim ("nut-free!") does not override
the ingredient panel. The ingredient panel is authoritative.
- A "may contain" warning flips the verdict to red whenever your
child's cross-contact stance says so — and the default stance for
any newly added allergen is "always flip red".
- The model never recommends medical action. The verdict is about
the packet, not about the child. Anaphylaxis treatment is decided
by your family's emergency action plan and your prescriber.
**Backend services this app depends on:**
- Auth: see section 4b
- Database: see section 4b
- Storage: see section 4b
- Email: see section 4b
- Payments: see section 4b (not used in v1)
- External APIs: see section 4b
**Environment variables you'll need to configure:**
- `GEMINI_API_KEY` — your Google AI Studio API key
- `FIREBASE_PROJECT_ID` — your Firebase project id
- `FIREBASE_SERVICE_ACCOUNT` — service-account JSON (server-side only)
- `RECALL_FEED_*` — optional, only if you want regulator-recall
alerts beyond the public feeds the app reads by default
**Cost + privacy notes:**
- One short paragraph per cost-sensitive capability: the verdict call
is the daily driver — about $0.012 per packet. A household scanning
10 packets a day spends about $3.60 a month on Gemini.
- One short paragraph on privacy: where the data lives (your Firebase
project), how to delete it (Settings → "Delete this child's profile
forever" and "Delete the whole household forever" — gone in 60
seconds), what is never sent for training.
**Documentation links:**
- AI Studio Build docs
- Gemini API multimodal, structured-output, multilingual docs
- Firebase Auth, Firestore, Firebase Storage docs
- A short note on the curated allergen database and how to extend it
for allergens not in the default set
**Accessibility:** same standards as the onboarding modal — focus
trap, `Esc`, ARIA, restored focus.
**Behaviour:**
- Always available — single click from anywhere in the app.
- Tooltip on the `(i)` icon: "How this app is built".
- Mobile: opens as a full-screen sheet that slides up.
- Should be the most honest part of the app — never hand-wave service
requirements; never say "AI" without naming the specific Gemini
model and capability.
## 6d. Detailed cost breakdown (deployer reads this BEFORE shipping)
- **Read packet + decide verdict (Gemini 3.5 Flash, medium thinking)** —
typical 1–2 images of a packet, ~600–900 input tokens for the
child profile + allergen database extract, ~400 output tokens for
the `LabelRead` JSON. ~$0.012 per scan.
- **Re-translate transcript (Gemini 3.5 Flash, low thinking)** —
typical 200 words of transcript, both directions. ~$0.0005 per
translation; runs only on demand (the parent picks a different
display language).
- **Find a safe alternative (Gemini 3.5 Flash, low thinking, grounded
search)** — ~$0.02 per query. The parent triggers this rarely.
- **Recall-alert check (Gemini 3.5 Flash, low thinking, grounded
search)** — runs daily over the household's last-60-day scan
history. ~$0.005 per household per day.
- **Generate pantry-orientation card (Nano Banana 2)** — ~$0.03 per
card. Generated once per child profile and cached; regenerated
only when allergens change.
- **TTS narration (Gemini 2.5 Flash TTS)** — billed per output token
(~$10/M output tokens), effectively ~$0.000003/character. A 30-
word verdict ≈ ~$0.0005 per spoken verdict. Used only when
accessibility is enabled.
- **Expected per-scan cost:** ~$0.012. **Heavy household scanning
20 packets a day:** ~$0.25 a day, ~$7.50 a month. **Light
household scanning 5 packets a day:** ~$0.06 a day, ~$2 a month.
- **Image storage:** Firebase Storage standard tier, ~$0.026/GB/
month. A typical packet photograph at 2048 × 1536 px is ~600 KB; a
household scanning 200 packets a month uses ~120 MB ≈ ~$0.004/
month. Pinned-pantry items kept forever; non-pinned scans pruned
after 90 days (user-configurable).
## 7. Design language
- **Mood:** A safety tool that lives on the phone of a tired parent.
Not a tech product. Not a clinical instrument. The supermarket
aisle at 5pm with the trolley, the kid in the seat, the strip
lighting, and the brain at 60% — the app's job is to make the
one decision the parent needs to make right now bigger than
anything else on the screen.
- **Typography:** A clean humanist sans for everything (Inter or
Geist). One display weight for the verdict reason short (semibold,
20–24px) and for the child's name in the verdict header. Small
caps avoided. No serif anywhere — this is not a heritage app.
- **Palette:**
- Background `#FFFFFF` for camera / verdict screens (maximum
legibility under fluorescent supermarket lighting).
- Verdict green `#1B873F` for the safe icon and safe-icon
backgrounds. WCAG-AA against white.
- Verdict red `#C0271C` for the unsafe icon and unsafe-icon
backgrounds. WCAG-AA against white.
- Neutral grey `#6E6E73` for secondary copy.
- Pale alert yellow `#FEF6D8` for the recall-alert banner (border
`#C29200`).
- Hard rule: green and red are never the only signal. The icon
weight (check is thick stroke, cross is thick stroke) carries
the signal independently of colour for colour-blind users.
- **Imagery:** The photograph of the packet the parent just took is
the hero of the verdict screen. Never replaced with an illustration.
Cropped only to show the trigger phrase, never tighter than the
parent's framing for the wider view. The pantry-card illustration
is the only generated image in the app and it lives outside the
decision flow.
- **Hand-feel touches:** The shutter button is a single large soft
circle that lights up green or red in the half-second after the
parent taps, before the verdict screen renders — so the parent
feels the result the moment they hear the click, before they read
it. The verdict screen lands with a single fade, no bounce, no
flourish — the decision is what the parent came for, not the
animation.
- **Spacing:** consistent 4-px base. Generous whitespace around the
big verdict icon — nothing else should compete with it.
- **Radius:** consistent token set (e.g. 8 / 16 / 28 px). Verdict
cards use 16; the shutter button uses 28; the avatar pills at the
top of the camera use 8.
- **Shadows:** subtle, layered, neutral grey. Avoid heavy drop-
shadows.
- **Motion:** purposeful — the shutter pulse, the verdict fade.
Respect `prefers-reduced-motion`. No bouncing splash animations.
No theatrical hero animations. The "why" panel slides up
intentionally; with reduced-motion it appears in place.
- **States:** every interactive element has hover, focus, active,
disabled. Loading uses skeletons where possible. Empty states
have helpful next-action guidance ("Tap to scan your first
packet"). The shutter button never disables — a parent at the
shelf cannot be told "wait".
## 8. Content generation rules
- Write **realistic, specific copy**. NO Lorem Ipsum. NO generic
placeholders like 'Your tagline here'.
- Invent plausible brand names, ingredient lines, may-contain
warnings, and trigger phrases that fit the domain (use the seed
content in section 8a as a starting point). When inventing,
lean on real-world packaging conventions — Bengali ingredient
panels typeset in Solaiman Lipi, Hebrew warnings with the kosher
symbol nearby, Vietnamese panels with vertical-Latin English
sub-translation — but never claim a fictional brand is a real
brand.
- Tone: calm, direct, child-named, free of corporate language. The
parent is exhausted; the words on screen need to be readable in
one glance.
- Headlines: punchy and concrete. No 'Empower your X' filler. No
'Revolutionize'. No 'Seamless'. No 'AI-powered'.
- Body copy: short paragraphs (1–3 sentences). Use lists where
appropriate.
- Plain language. Avoid jargon — except where the user already
speaks the jargon (the newly-diagnosed family wants to see
"epinephrine" not "EpiPen"; the dietician user wants to see
"FPIES" if that's their child's diagnosis).
- Where the app outputs AI-generated content (the verdict, the
translation), never label it as "AI says" — let it speak as the
app speaks. Use small uncertainty cues only where epistemic honesty
requires them (an unsafe_ambiguous verdict shows the trigger as
"natural flavour" with a small underline; tapping reveals "this
could legally contain peanut in this jurisdiction").
## 8a. Seed content (use these specific examples)
Anchor every generated copy + sample data point in the concrete content
below. Use these names, numbers, dates, and snippets verbatim where
helpful, or generate close variants that sit in the same world.
**Demo household (sidebar):**
- **The Akhtar household.** Aarav, 6, peanut + tree nuts (cashew,
almond, hazelnut, walnut). Cross-contact stance: always-flip-red
on may-contain. Phonetic: "AAH-rahv". Diagnosed: 2024-09-15.
Allergist: Dr Patel, Royal London. School nurse: Mrs Williams,
Whitefield Primary.
- **The Ben-Ari household.** Noor, 4, cow's-milk protein. Cross-
contact stance: always-flip-red on shared-facility. Phonetic:
"NOOR". Diagnosed: 2023-11-02. Allergist: Dr Cohen, Tel Aviv
Sourasky. School nurse: Mrs Levi, Gan Shalom.
- **The Nguyễn household.** Linh, 8, shellfish (crustacean + mollusc).
Cross-contact stance: always-flip-red on may-contain. Phonetic:
"LIN" (short i). Diagnosed: 2022-04-30. Allergist: Dr Tran,
SickKids Toronto. School nurse: Mr Singh, Yorkmills Junior.
- **A generic single-allergen household.** Mehmet, 7, sesame. Cross-
contact stance: always-flip-red on may-contain. Phonetic:
"MEH-met".
**Sample verdict in detail view (this is what the demo should show):**
- **Child:** Aarav (peanut, tree nuts)
- **Product name (verbatim):** "Crunchy Choco Chip Cookies"
- **Brand (verbatim):** "Higgleby's"
- **Barcode:** "5012345678901"
- **Source languages detected:** ["en"]
- **Packet image URIs:** front-of-pack, back-of-pack
- **Ingredient panel found:** true
- **May-contain warning found:** true
- **Shared-facility warning found:** false
- **Transcript block — ingredient_list (verbatim):** "Wheat flour,
sugar, vegetable oil (palm, sunflower), chocolate chips 15% (sugar,
cocoa mass, cocoa butter, soya lecithin, vanilla), invert sugar
syrup, glucose-fructose syrup, raising agents (sodium hydrogen
carbonate, ammonium hydrogen carbonate), salt, natural flavouring."
- **Transcript block — may_contain_warning (verbatim):** "May
contain traces of peanut, milk and egg."
- **Matched allergen (1):**
- allergen_id: "peanut"
- matched_term_verbatim: "peanut"
- matched_term_source_language: "en"
- matched_term_translation: "peanut"
- match_kind: "may_contain_warning"
- match_confidence: 0.98
- position_on_packet: "may-contain warning, line 1"
- **Verdict:** unsafe_may_contain
- **Verdict reason short:** "Not safe for Aarav. May contain peanut."
- **Verdict reason long:** "The packet's allergen warning says 'May
contain traces of peanut, milk and egg.' Aarav's profile is set to
treat any 'may contain peanut' warning as not safe. Try a different
brand or check with staff about the production line."
- **Trigger term:** "peanut"
- **Recommended next action:** "scan_another_packet"
- **Reading confidence:** 0.97
**Sample input artefacts (for the build to demonstrate):**
- A Higgleby's Crunchy Choco Chip Cookies packet in English with a
"may contain peanut" line on the back of pack — verdict:
`unsafe_may_contain` for Aarav, `safe` for Noor.
- A Sabra-style hummus tub labelled in Hebrew with a "ייתכן שמכיל
חלב" (may contain milk) warning — verdict: `safe` for Aarav,
`unsafe_may_contain` for Noor.
- A Korean-Vietnamese instant ramen packet with "건새우 분말" (dried
shrimp powder) listed as the third ingredient — verdict: `safe`
for Aarav, `unsafe_contains` for Linh.
- A Portuguese-labelled sesame-seed bread roll from Pingo Doce, with
"contém sementes de sésamo" prominent — verdict: `safe` for
Aarav, `unsafe_contains` for Mehmet.
- An unbranded paper bag of trail mix with no ingredient panel —
verdict: `unsafe_unreadable` for every child profile, with reason
"no ingredient panel visible".
- A snack bar whose ingredient panel says "natural flavour" without
specifying source, in a jurisdiction where natural flavour may
legally contain peanut — verdict: `unsafe_ambiguous` for Aarav,
with the trigger highlighted as "natural flavouring".
**Sample voice copy:**
- Onboarding: "Point the phone at any packet. We'll tell you in 3
seconds whether your kid can eat it."
- Camera prompt: "Scanning for **Aarav** (peanut, tree nuts). Hold
the packet flat and tap to scan."
- Processing: "Reading the panel…" / "Checking for peanut…" /
"Looking for 'may contain' warnings…"
- Empty history: "No scans yet. Tap the shutter to scan your first
packet."
- Verdict — red (contains): "Not safe for Aarav. Contains peanut
oil."
- Verdict — red (may contain): "Not safe for Aarav. May contain
peanut."
- Verdict — red (unreadable): "Couldn't read the panel clearly
enough. Try again with brighter light and the label flat."
- Verdict — red (ambiguous): "Not safe for Aarav. Ingredient
'natural flavouring' could contain peanut — check with the brand
or skip."
- Verdict — green: "Safe for Aarav. No peanut. No 'may contain'
warning."
- Primary action button under every verdict: "Scan another packet"
- Save confirmation: "Saved to Aarav's history. Higgleby's Crunchy
Choco Chip Cookies — not safe."
- Recall banner: "A brand you've scanned for Aarav has been recalled.
Tap to see the regulator notice."
- Pantry empty: "No safe packets pinned for Aarav yet. Scan one and
swipe up to pin it."
**Sample babysitter share email subject + body:**
- Subject: "For tonight — Aarav's food list"
- Body: "Hi Sarah — Aarav has a severe peanut and tree nut allergy.
Here's the live list of what's safe for him in our pantry, and
what's not. If you're unsure about anything else, scan it with the
app — link below. His epinephrine is in the top drawer in the
hallway. Emergency: 999, then me." [Open the live list] [Open the
scanner]
**Sample sample-archive sidebar entries (the demo household's recent
scans):**
- "Higgleby's Crunchy Choco Chip Cookies — Aarav — not safe (may
contain peanut) — 2 May 2026"
- "Tnuva yogurt 3% — Noor — not safe (contains cow's milk) — 1 May
2026"
- "Pingo Doce sesame bread roll — Mehmet — not safe (contains
sesame) — 30 Apr 2026"
- "Sabra-style hummus 250g — Aarav — safe — 30 Apr 2026"
- "Lotus oat bar — Aarav — safe — 28 Apr 2026"
## 9. Media & assets
- **Hero image (landing screen):** A photographed-looking shot of a
dad's hand holding a phone over a chocolate biscuit packet on a
supermarket shelf, the screen of the phone showing a big green
check. Generate via Nano Banana 2 with a prompt emphasising "warm
supermarket lighting, a real packet in a man's hand at biscuit-
shelf height, the phone's screen visible with a big green check,
no faces in frame, no readable brand on the packet".
- **App icon / wordmark:** A bold sans-serif wordmark with a small
circle behind it that contains either a check or a cross
silhouette, switching with the system theme.
- **Empty-state illustration:** A simple line drawing of a packet
with an ingredient panel and a small dotted line pointing to the
may-contain warning. Hand-drawn aesthetic, not a flat icon.
- **Demo packet photographs:** Generated per the prompts in section
8a — Nano Banana 2 prompts that specifically request "a packet of
chocolate biscuits on a supermarket shelf, the back of pack
visible, the ingredient panel readable, an 'allergen warning' box
visible, warm overhead lighting, no people in frame". Each demo
packet should look photographed, not rendered.
- **Pantry-orientation card:** Generated by the per-call image
generation prompt in section 4b. Per child, A5 at 300 dpi, warm
pastel palette, hand lettering.
- **Stock fallbacks:** If image generation fails, fall back to the
photographed sample packet from `/public/samples/sample-packet.jpg`.
Never to a "🍪" emoji.
- **Generated imagery:** prefer Nano Banana 2 over stock photography.
Prompt for warmth, asymmetry, and slight imperfection — avoid the
glossy 'AI render' look.
- **Optimisation:** WebP/AVIF, `loading="lazy"`, explicit `width`/
`height` to prevent layout shift.
- **Icons:** `lucide-react` for UI. Use sparingly — never decorative-
only. The check and cross icons used on the verdict screen are
custom-drawn with extra-thick stroke weight so they are
recognisable independent of colour.
### Build-time asset manifest (explicit specs)
Every image, illustration, and visual reference mentioned above must resolve to ONE of the three buckets below — runtime-generated, seed-shipped, or user-supplied. Do NOT ship `` tags whose `src` is not listed here. Do NOT depend on bare "section 8a prompts" without binding them to explicit paths and model IDs.
**Bucket 1 — Runtime-generated (Nano Banana Pro `gemini-3-pro-image` for hero/demo photographs; Nano Banana 2 `gemini-3.1-flash-image` for in-app illustrations and reference-conditioned variants).** Cached to Firebase Storage; served via signed URL. Every reference above to "Nano Banana 2" or "Nano Banana Pro" MUST be wired to one of these specific calls with an explicit model id:
- `/public/generated/hero.webp` (2400×1500, WebP) — model `gemini-3-pro-image` — uses the literal prompt described as "Hero image (landing screen)" above. Run once at build; commit a `/public/samples/hero-fallback.webp` (1600×1000) generated from the same prompt with `gemini-3.1-flash-image` so the page renders if quota is exhausted.
- `/public/generated/demo/{demo-slug}-{NN}.webp` (1600×1200, WebP) — model `gemini-3.1-flash-image` (reference-conditioned where the prior frame is passed as input) — one path per "Demo X" image referenced above. The slug derives from the seed example in section 8a; the NN index covers each frame in the demo sequence.
- `/public/generated/illustrations/{name}.webp` (1024×1024, WebP) — model `gemini-3.1-flash-image` — one path per named illustration above ("Empty-state illustration", "Recipe-card hero illustrations", "Curriculum picker imagery", "Period-style frames", etc.). Each illustration's prompt is the literal description above; ship a deterministic seed in the request so re-runs are reproducible.
**Bucket 2 — Seed assets shipped with the deliverable.** Every "Stock fallback" path referenced above (e.g. `/public/samples/sample-X.jpg`) is generated once via Nano Banana 2 (`gemini-3.1-flash-image`) at 1024×1024 WebP using the same prompt as its Bucket-1 counterpart, then committed to the repo so the page renders identically if Gemini quota is exhausted or the user is offline. Replace any `.jpg` extension above with `.webp` to match the optimisation rule. Also commit these empty-state seeds (1024×1024 WebP, single-stroke hand-drawn line, no colour fill):
- `/public/samples/empty-state-primary.webp` — line drawing of the app's primary empty surface (the named "Empty-state illustration" above), generated from that exact prompt.
- `/public/samples/empty-state-archive.webp` — line drawing of an empty saved/archive view, single-stroke outline.
- `/public/samples/empty-state-error.webp` — line drawing of a hand placing a single object aside with care, used when an AI call fails.
**Bucket 3 — User-supplied.** Uploads from the user's camera / file picker land at the Firebase Storage path conventional for this template (named in section 4b). The build ships with Bucket-1 + Bucket-2 only; no user-supplied images at first paint.
**Hard rules**
- Every `` tag MUST have a `src` that resolves to a path listed in Bucket 1, Bucket 2, or a Bucket 3 upload path. Anything else is a build error.
- No bare `image.jpg` / `hero.jpg` / `placeholder.png` references anywhere in the code.
- Model IDs: `gemini-3-pro-image` for hero-quality photographic generation; `gemini-3.1-flash-image` for in-app illustrations, reference-conditioned variants, empty-state seeds, and stock fallbacks. Never use a legacy model id (no `imagen-*`, no `gemini-1.5-*-image`).
- File format: WebP everywhere (AVIF acceptable where the target browsers support it). No `.jpg` / `.jpeg` / `.png` in `/public/samples/`.
## 10. Interactivity & states
- Every interactive element has hover, focus, active, and disabled
states.
- The shutter button is the most important interactive element in
the app. It is large (≥ 80 px), centrally located, and never
disabled. Tapping it always captures, even if a prior verdict has
not finished rendering — the new capture overrides the old.
- Forms validate inline and show specific error messages (not
"Invalid input"). The allergen multi-select is grouped by allergen
family (tree nuts as a group, with individual ticks for cashew,
almond, hazelnut, walnut, etc.).
- Loading states use skeletons that match the eventual layout, not
spinners. The verdict screen's "thinking" state is the photograph
the parent just took, with a thin sweep animation from top to
bottom signalling "reading the panel" — never a generic spinner.
- Empty states explain the next action with a button whose label
fits THIS app's domain: "Scan your first packet", "Pin a safe
snack to Aarav's pantry", "Add a child profile" — never a generic
"Add your first item".
- All AI-generated content is rendered the moment it arrives; the
verdict icon appears as soon as the schema's `verdict` field is
parsed, before the verbatim transcript finishes streaming.
- If an AI call fails or times out, the verdict is **red** —
`unsafe_unreadable` with reason "couldn't reach the model in time —
try again, or skip this packet". The verdict is never delayed
silently. The shutter button is available immediately for a retry.
- The verdict screen never auto-dismisses. The parent always taps
"Scan another packet" themselves — the app does not decide when
the parent is done reading the reason.
- The map view (not applicable to this app) is not present. The app
has no map.
- The verdict screen respects `prefers-reduced-motion`: the icon
fade happens in 200 ms by default; with reduced motion the icon
appears in place.
## 11. Tech & responsive requirements
- **TTS markdown-stripping preprocessor:** before sending any user-authored markdown to `gemini-3.1-flash-tts-preview`, strip non-spoken markdown: `#`/`##`/`###` headings (keep the title text), `**bold**` (keep the inner text), `[label](url)` (keep `label`, drop URL), `` ``` `` fenced code blocks (skip entirely), `>` block-quote markers (keep the text), and `|` table pipes (read row-by-row as sentences). Insert `…` between sentences for a short pause and a blank line plus `—` between paragraphs for a long pause. The model does not understand markdown; raw markdown will be read aloud as literal characters ("asterisk asterisk").
- **File downloads on Safari / Firefox:** when offering local-disk save of any export (PDF, CSV, MP3, ZIP, JSON, image), fall back to `` with a blob URL — the File System Access API (`showSaveFilePicker()`) is Chromium-only. Detect with `'showSaveFilePicker' in window`; otherwise use the anchor-download path.
- **Stack:** React + TypeScript + Tailwind CSS. Functional components
+ hooks. Use Shadcn UI primitives where appropriate.
- **Build runtime:** AI Studio Build — full-stack with Cloud Run
server-side functions. All Gemini API calls happen server-side; API
key lives in Secrets Manager, never in client bundle.
- **Model selection:** explicitly pin `gemini-3.5-flash` for the verdict
call (the heart of the app) and `gemini-3.5-flash` for translation,
recall-alerts, and follow-up suggestions. Set `thinkingLevel`
explicitly per call. Use `gemini-3.1-flash-image` for the pantry-
orientation card and `gemini-3.1-flash-tts-preview` for the verdict
readout. Do NOT use `gemini-3.5-flash` or `gemini-3.1-flash-tts-preview` —
those are not valid model IDs.
- **Database:** Firestore (auto-provisioned by AI Studio Build). Show
the demo household on first launch.
- **Auth:** Firebase Auth — Google sign-in by default; Apple sign-in
next to it; magic-link email as fallback. The babysitter share
uses magic-link.
- **Storage:** Firebase Storage for packet photographs. Pre-signed
URLs only. Manual enable in the Firebase console required.
- **Mobile-first.** This app is used standing up in a supermarket
aisle. Verify layouts at 375 px (iPhone SE), 768 px (iPad), 1024
px, 1440 px+. The verdict icon must be readable from 60 cm at the
most-common adult viewing angle.
- Use `clamp()` for fluid typography. Prefer container queries over
media queries for component-level responsiveness.
- Use `dvh` / `svh` instead of `vh`. Respect safe-area insets on iOS
(the home indicator must not occlude the "scan another packet"
button).
- Zero horizontal overflow at any width. Zero layout shift on load.
- Persist user data in Firestore. Use real-time listeners on the
pantry view so both parents see the same list.
- Optimistic UI on writes; reconcile on response. The verdict itself
is NOT optimistic — the parent waits for the model. Showing a
speculative verdict before the model returns would defeat the
safety stance.
- Camera flow uses the Web Camera API with continuous focus where
supported; falls back to native camera otherwise. AE/AF lock on
ingredient-panel rectangle.
- **iOS Safari gotchas (graceful degradation):** camera permission
does NOT persist across page reloads on iOS — re-request on every
open and show a friendly explainer if denied. Backgrounded Safari
tabs pause `getUserMedia` — checkpoint the last frame and re-acquire
the stream on `visibilitychange === 'visible'`. Camera capture may
degrade resolution or torch availability on low-battery / Low Power
Mode — if `MediaTrackCapabilities.torch` is absent or
`MediaStreamTrack.getSettings()` returns reduced resolution, show a
one-line "low-light scan — hold steady" hint instead of a hard fail.
Always offer a `capture="environment"` ``
fallback so a scan still works when WebRTC is denied.
- Audit log writes are server-side, transactional, and never deleted
by the user — the parent can hide a scan from their history but
the audit log retains it for the household's safety.
## 12. Accessibility (WCAG 2.2 AA)
- Semantic HTML — `header`, `nav`, `main`, `section`, `article`,
`footer`.
- All interactive controls reachable by keyboard with a visible focus
ring.
- Color contrast ≥ 4.5:1 for body, 3:1 for large text and UI
components. The verdict green and red are verified against the
white background at AA.
- All images have meaningful `alt` text. The packet photograph on the
verdict screen has `alt` describing the artefact ("photograph of a
Higgleby's chocolate biscuit packet, back of pack, ingredient panel
visible").
- Form fields have associated `