================ ONE-SHOT BUILD CONTRACT (read first) ================ Build this in Google AI Studio "Build" in ONE shot — a complete, working app, no follow-up turns. These are hard rules, not suggestions: 1. TARGET = Full-Stack Web (Node server runtime, secrets, Firebase allowed). If you target Native Android instead, you MUST drop all server/DB/Workspace/ secrets and persist locally (Room / SharedPreferences) only. 2. PIN EVERY MODEL CALL — never let the agent auto-select (it downgrades on quota): - Reasoning / text -> gemini-3.5-flash (thinkingLevel: minimal|low|medium|high) - 4K image + legible text -> gemini-3-pro-image (image_size "4K", up to 14 refs) - High-volume image -> gemini-3.1-flash-image - Expressive TTS -> gemini-3.1-flash-tts-preview (inline tags e.g. [whispers]) - Realtime audio/video (WebSocket) -> gemini-3.1-flash-live-preview - Sandboxed agent -> antigravity-preview-05-2026 3. DIVISION OF LABOR — the model ONLY parses/extracts to a strict responseSchema. ALL math, money (store currency as integer minor units / cents), sorting, balancing and graph logic run in deterministic TypeScript/Python. The model must never compute totals, splits or balances itself. 4. responseSchema sanitation — no regex patterns, no fixed-length tuples, no format validators in the schema (they crash the OpenAPI engine). Enforce those in server-side code AFTER parsing the JSON. 5. responseSchema and google_search grounding are MUTUALLY EXCLUSIVE in one call. 6. CODEGEN — split large output into modular, single-responsibility files so no file is truncated by the output-token cap. 7. Every external call gets a graceful fallback (e.g. manual paste if a Workspace read fails). Never a silent dead end. 8. ROBUST STORAGE & CANVAS — Wrap all `localStorage`/`sessionStorage` operations (especially JSON parsing and writes) in `try-catch` blocks to prevent crashes in private windows or quota overflows. Canvas drawing elements must dynamically handle window resize and scale pixel density (`window.devicePixelRatio`) to avoid blurry graphics on retina displays. ===================================================================== # MUST OBEY — Mobile-first build requirements This app's PRIMARY surface is a mobile phone. Build it impeccably on mobile FIRST, then verify on tablet and desktop. Treat the rules below as non-negotiable hard constraints, not suggestions. ## Viewports to verify (every screen, every state) - 320 px, 360 px, 375 px, 390 px, 414 px, 480 px - 768 px, 834 px (iPad portrait / Pro 11) - 1024 px, 1280 px, 1440 px, 1920 px, 2560 px - Plus: 200% browser zoom, landscape orientation on every mobile width, iPhone with safe-area insets visible ## Hard layout rules - Mobile-first CSS. Default styles target mobile; `@media (min-width: ...)` for larger viewports. - Use `dvh` and `svh` instead of `vh` for full-height surfaces (iOS Safari URL-bar bug). - Use `clamp()` for fluid typography across all viewports. - Prefer container queries (`@container`) over media queries for component-level responsiveness. - Use `min(100%, ...)` widths so content never overflows. Zero horizontal overflow at any viewport. - Add `` to every page. - Apply `padding: max(safe-area-inset-X, fallback)` on every edge-bleeding container so notched iPhones in landscape never clip content. - Wide tables and code blocks scroll INSIDE their container (`overflow-x: auto`), never push the body. - Use `background-attachment: scroll` on mobile, not `fixed` (iOS Safari repaint bug). - Avoid `backdrop-filter` on animated elements. Use it sparingly on static surfaces only. - **Canvas Scaling**: Canvases must dynamically scale with window resize events and properly handle high-DPI screens (`window.devicePixelRatio`). Set physical dimensions (`canvas.width`/`canvas.height`) using pixel ratio and render relative to this grid, using CSS to control responsive viewport scaling. - **Robust Storage**: Every access to `localStorage`/`sessionStorage` (especially `JSON.parse` of loaded state or writes) MUST be wrapped in a `try-catch` block to handle disabled storage, private browsing mode, quota limits, or corrupted JSON gracefully. Fall back to a robust in-memory object store. ## Touch & accessibility - Tap targets ≥ 44 × 44 px on touch (Apple HIG). Increase to 48 px under `@media (hover: none) and (pointer: coarse)`. - All interactive controls reachable by keyboard with a visible focus ring; respect `:focus-visible`. - Color contrast ≥ 4.5:1 for body text, 3:1 for UI components. - All images have meaningful `alt`. Decorative images use `alt=""`. - Respect `prefers-reduced-motion: reduce` — zero animation durations under that query. - Forms validate inline; error messages are specific, not "Invalid input". - Modals: focus trap, `Esc` closes, `role="dialog"`, `aria-modal="true"`, focus restored on close. ## Performance bar (Lighthouse mobile, throttled 3G/4G) - LCP < 2.5 s · INP < 200 ms · CLS < 0.1 - JS bundle gzip < 200 KB mobile-first; lazy-load non-critical screens via `React.lazy` / dynamic imports. - No render-blocking resources above the fold. - Images: WebP/AVIF preferred, `loading="lazy"`, explicit `width`/`height` attributes (zero CLS), `srcset` for retina. - Videos: `preload="metadata"`, low-resolution poster, max 720p mobile fallback. Never autoplay with audio. - Fonts: `font-display: swap`; preload only the one used above the fold. - Smooth scroll honoured via CSS `scroll-behavior: smooth` with reduced-motion fallback. ## Pre-ship mobile checklist (the deployer MUST verify before declaring done) 1. Open at 375 px in DevTools — every screen scrolls vertically only; zero horizontal scroll. 2. Browser zoom 200% — layout reflows without overlap. 3. iPhone Safari with the URL bar visible AND landscape — no content under the home indicator; no notch clipping. 4. iPad portrait (768 px) and landscape (1024 px) — no awkward gaps; tablet-specific breakpoints land cleanly. 5. Tap every interactive element with a thumb at real-device size — every target is easy to hit. 6. `prefers-reduced-motion: reduce` — every transition / animation skips cleanly, scroll-behavior becomes instant. 7. Lighthouse mobile score ≥ 90 across all 4 categories. 8. Zero `console.error` and zero CLS shift in real-device testing on a mid-tier Android (e.g. Pixel 6a) and an iPhone SE. --- The original template starts below. All rules above apply on TOP of whatever this template specifies. --- # Kitchen Phrasebook ## 1. Project **Kitchen Phrasebook** is a live-translation cooking companion for families whose elders speak one language and whose youngest cooks speak another. The grandmother — a paati, a lola, a yiayia, an abuela, a nai-nai — stands at the stove speaking the way she has always spoken, and her grandchildren hear the recipe in English in real time, paced so she can pour the oil without missing a line. The recipe is the grandmother's recipe: nothing is substituted, nothing is "simplified for Western kitchens", and the dish she learned from her own mother is the dish that ends up on the table. This is the kind of app a Sydney-born brother and sister build on a Saturday afternoon — because their nai-nai has flown in from Guangzhou for six weeks, she has decided today is the day she teaches them her ginger-and-spring-onion steamed fish, and the kids' Cantonese stops at "hello" while her English stops at "rice". It is also the kind of app a Tamil-Brit grandson sets up on the counter when his paati arrives from Chennai to teach him the kuzhambu her own paati taught her, or the kind a Salvadoran-American granddaughter sets up the morning her abuela in the Mission District decides she will finally show her how to fold a real pupusa. Same shape of moment, different kitchen, different language. The single demo that proves the magic: open the app, place the phone on the counter, tap **Start cooking together**, and pick the grandmother's language. She begins: _"Yāt tìuh sàn sīn yú, mh hóu tàai daaih…"_ — a fresh fish, not too big. The English appears on the big tile within a beat and a half, large enough for two kids to read across the counter: _"One fresh fish, not too big."_ The app does not interrupt her to ask for clarification. It does not cut her off when she pauses to debone. When she stops talking because she is concentrating on slicing ginger, the English tile holds the last instruction so the grandchildren can keep working while she works. And in the harder cases — a paati who slips between Tamil and English mid-sentence, a lola whose Cebuano dialect changes every time she is excited, a babushka whose Russian is half Yiddish endearment — the app does not flatten the mix. The English tile shows the dominant translation; a small chip below shows the inserted phrase in its original ("_appa_ — father", "_Tato_ — papa") so the grandchild learns that the word their grandmother just used for father is the word their parent grew up hearing. The dish is never edited. _Ghee_ stays _ghee_. _Chhatak_ stays _chhatak_, with a parenthetical gram conversion the first time it appears and never again. _Suka_ stays _suka_. _Maggi cube_ stays _Maggi cube_. **Tagline:** _Cook your grandmother's recipe — in any kitchen, any language, paced so she never has to repeat herself._ ## 2. Target audience - Diaspora grandchildren, ages 8–35, cooking with a grandparent who flew in for a long visit — Cantonese, Hokkien, Mandarin, Tagalog, Vietnamese, Korean, Tamil, Telugu, Malayalam, Hindi, Urdu, Bengali, Punjabi, Sinhala, Khmer, Thai, Lao, Burmese, Amharic, Tigrinya, Swahili, Yoruba, Igbo, Twi, Arabic (Levantine, Egyptian, Maghrebi), Farsi, Pashto, Dari, Turkish, Kurdish, Armenian, Greek, Polish, Russian, Ukrainian, Portuguese (Brazilian and European), Spanish (Caribbean, Andean, Mexican, Castilian), Quechua, Haitian Creole, Yiddish - Grandparents who would never use an app on their own but will tolerate "the phone the kids put on the counter" for one afternoon - Adult children translating in the middle — the parent generation who grew up speaking the language at home and now wants their own kids to inherit the recipe without the dilution of going through them - Parents of bicultural toddlers and primary-school kids — using the app as a Saturday morning routine for the heritage language to enter the kitchen even when the grandparent is on another continent - Adult heritage-language learners (third- and fourth-generation diaspora) cooking from their grandmother's voice memos for the first time, with no fluent intermediary still alive - Cooking-school students paired with a guest elder for a single class — community kitchens, Saturday "abuela classes" at the community centre, refugee-resettlement kitchens - Couples in mixed-language households where one partner's mother does not speak the household's lingua franca - Adoptees and reunion families cooking with a recently-found birth grandparent for the first time - Hospice and palliative-care families recording a grandparent cooking their signature dish one last time — the dish, the words, the pacing, all preserved as an heirloom ## 3. Core value propositions Surface these clearly through copy, visual emphasis, and section ordering — they are the reasons families pick this app over Google Translate or a generic captioning tool. - **Live, two-way, cooking-paced.** Gemini Live API streams the grandmother's speech and produces translated text and TTS within a beat — fast enough to follow, slow enough that she does not feel rushed. When she pauses for thirty seconds to debone the fish, the app holds the last line on the big counter tile and waits. It does not summarise. It does not interrupt. - **The grandmother's recipe stays the grandmother's recipe.** The app translates her words; it does not substitute her ingredients. _1 chhatak ghee_ is rendered as _1 chhatak ghee (≈ 60 g)_ on first appearance and as _1 chhatak ghee_ thereafter. _Maggi cube_ is _Maggi cube_, not "stock cube". _Pak choi from the corner shop_ is _pak choi from the corner shop_, not "bok choy (Chinese cabbage)". When she says _"a fistful of dried shrimp"_, the English tile says _"a fistful of dried shrimp"_, not _"about ¼ cup of dried shrimp"_. - **Dialect-aware, not Mandarin-by-default.** Cantonese is not auto-corrected into Mandarin. Cebuano is not flattened to Tagalog. Maghrebi Arabic is not assumed to be Levantine. Calabrian Italian does not become Standard Italian. The grandmother's exact dialect is detected at the start of the session and the chip shows it so the grandchildren can ask her: _Paati, is this how your mother said it too?_ - **Code-switching is preserved, not erased.** When she slips between languages — Tamil into English for "pressure cooker", Cantonese into Mandarin for the dish name, Spanish into K'iche' for the corn — the app keeps both layers visible. The inserted phrase appears in its original alongside the translation. The grandchildren learn the word in the dialect their grandmother actually uses. - **Voice presence: she pours the oil without missing a line.** TTS pace is matched to cooking, not to podcasts. The English audio (when on) speaks at about 110 words per minute with explicit pauses at line breaks so the grandchildren can keep cutting while it reads. The audio never overlaps with her ongoing speech; it waits for her to finish before speaking. - **The recipe writes itself.** Every session is automatically structured into a recipe card: ingredients with the verbatim units, steps numbered, the grandmother's specific tips preserved verbatim ("_paati says: never let the curry leaves brown — pull them out the moment they hiss_"). The card is printable. The card is shareable with the cousins in another city who could not be there. - **Her voice is the heirloom.** With the grandmother's recorded consent, the session audio is preserved at full quality, indexed by step, and replayable forever. Years later, the grandchild can press play on step three and hear their grandmother again saying _"now the garlic"_ in her own voice. The recipe card links straight to the moment. - **Private by design.** Everything is per-family. No public recipes feed, no AI training, no shared learnings across households. The grandmother's voice is the grandmother's voice — used only inside this family's archive. ## 4. Features to build - "Start cooking together" big primary CTA on the home screen — single tap, no setup gauntlet - Language picker on first launch with locale-and-dialect granularity (Cantonese (Guangzhou), Cantonese (Hong Kong), Cantonese (Toishan), Tamil (Chennai), Tamil (Jaffna), Spanish (Salvadoran), Spanish (Castilian), Arabic (Levantine), Arabic (Egyptian), Arabic (Maghrebi), …) - Live API session: real-time audio in (grandmother's voice) → real-time translated text out (on the big counter tile) → optional translated TTS out (off by default; can be turned on by the grandchild) - "Counter mode" UI — full-screen tile, two-finger swipe to scroll history, no time-outs while the session is live, screen kept awake - Code-switch detection — the inserted phrase appears under the main translation with the original language tagged ("_appa_ · Tamil → father") - Verbatim ingredient capture — units stay in the source language ("_1 chhatak ghee_", "_2 anjili_", "_un puñado_", "_una cucharada colmada_"), with a parenthetical gram/ml/teaspoon conversion appended only on first mention - Auto-segmented recipe builder — every prep, every step, every tip is parsed into a structured `RecipeStep` as she talks; the user can review and re-order at the end without losing her exact words - "Paati's note" chips — short asides, tips, and warnings the grandmother makes ("_be careful, the oil splashes here_") are extracted as `tips[]` and preserved verbatim with translation - Pause detection — when the grandmother stops speaking for more than four seconds, the app does not start filling silence with text; the last line stays on the tile - Step-by-step photo capture — the grandchild can tap the screen at any moment to capture a photo of what the dish looks like at that step; the photo attaches to that step in the recipe - "Replay this moment" — every step links back to the audio segment where she said it; the grandchild can press play and hear her voice years later - Multi-dish session — one grandmother often teaches three dishes in one afternoon ("rice, fish, then the soup"); the app auto-detects dish boundaries and starts new recipes when she signals one - Family share — invite the cousins in another city; they receive the recipe card + the audio replay; they can add their own notes ("paati used to add a little more sugar") - Heirloom mode — at session end the grandchild is offered: "Save this as a family heirloom?" — full audio + transcript + recipe card archived with the date and the dish name; the grandmother is asked once for spoken consent at the start of the session - Printable recipe card export — typeset, A4 or US letter, with the dish name in both languages, the grandmother's tips in their own panel, and a QR code to her voice on the bottom - Pre-session glossary upload — the family can optionally drop a list of the names their grandmother uses for specific things ("she calls it _ammini_, not _ammuli_") so the model uses her preferred terms from the start - Offline-fallback transcript — if the kitchen Wi-Fi drops mid-session, the app records the audio locally and processes it the moment connection is back; the grandmother is not asked to repeat - "Hand back" gesture — a small button the grandchild presses to tell the model "stop translating now, she's just talking" — for the side-stories the family does not need on the recipe card (the gossip about Auntie So-and-So in between stirring) - Voice-clone option (opt-in, consent-gated) — once she has spoken for an hour total across sessions, the family can optionally clone her voice for a small heirloom moment (the grandchild's own kid hearing great-paati read the recipe aloud one day) — never on by default - Recipe lineage tree — link the dish to "the way paati's mother made it", "the way I make it on Sundays", "the way my mother made it for Diwali" — chained voice memos across generations ## 4b. Required Gemini capabilities + backend services **This template's intelligence comes from the Gemini capabilities below. Wire them up explicitly — don't substitute generic LLM calls.** ### Gemini capabilities (the load-bearing intelligence) - **Gemini Live API (real-time audio-in, text-and-audio-out)** — the heart of the app. Bidirectional streaming session, source-language audio in, translated text out on the counter tile within ≤ 1.2 seconds, optional translated TTS out paced for cooking. Voice activity detection on the input side; pause-detection thresholds tuned for the kitchen (four seconds of silence does not mean "session over"; it means "she is concentrating on the knife"). - **Multilingual translation with dialect awareness** (Live model + Gemini 3.5 Flash structured pass) — Cantonese vs Mandarin vs Toishan vs Hokkien; Cebuano vs Tagalog vs Hiligaynon; Salvadoran Spanish vs Mexican vs Castilian; Levantine vs Maghrebi vs Egyptian Arabic; Chennai vs Jaffna Tamil. Dialect is pinned in the session config and propagated to every Gemini 3.5 Flash structured-extraction call. - **Cooking-paced TTS** (`gemini-3.1-flash-tts-preview`) — when the grandchild enables English audio, the translated text is read at ~110 WPM with explicit ellipsis pauses at sentence ends and blank-line / em-dash pauses at step boundaries. Pronunciation comes from the chosen voice's native locale; mid-utterance voice switching is not supported, so the grandmother's verbatim inserted phrase ("_ghee_", "_pak choi_") is left in the English voice and rendered as the loanword that English already accepts. - **Structured output / JSON Schema** (Gemini 3.5 Flash) — at session end (and at every detected dish boundary mid-session), the Live transcript is passed to a Gemini 3.5 Flash structured-extraction call that returns the typed `Recipe` schema below. Translation is done here in batch for the printable card; the Live session's translations are for in-the-moment kitchen reading. - **Long context (1M tokens)** — multi-dish three-hour sessions (paati cooks three things; lola tells stories between dishes; abuela alternates with cousins) fit comfortably under 1M tokens for both audio-transcript chunks and downstream extraction. Guardrail: at ≥ 90 minutes of session audio, split at detected dish boundaries before passing to the structured-extraction call. - **Image input for the dish-confirmation pass** (Gemini 3.5 Flash multimodal) — at step end, when the grandchild taps the screen to capture what the dish looks like, the photo is sent to Gemini 3.5 Flash along with the step transcript: "does this look like what the grandmother just described?" If not, surface a soft chip on the step ("paati said _crisp brown crust_; the photo looks pale — keep going?"). Never overrides her. - **Nano Banana 2** (`gemini-3.1-flash-image`) — used only for the optional hero illustration on the printable recipe card, in a warm, watercolour-leaning style. Not used for the dish itself in the in-app flow (the photos the grandchild took are the canonical images). - **Thinking levels** — `medium` for the structured-extraction call at session end (it has to disambiguate dish boundaries, code-switches, and dialect markers). `low` for the per-step parsing during the Live session. Live API does not expose `thinkingLevel` — pacing is controlled via the session config and the system instruction. ### Backend services - **Auth — Required.** Firebase Auth with Google sign-in (auto-provisioned by AI Studio Build). **Apple sign-in is optional but user-configured**: it requires an Apple Developer account, Service ID, Key ID, and private key wired into the Firebase Auth console. **Magic-link email** (used for family invitations) requires the sender domain to be authorised in Firebase Auth before the first invitation is sent. - **Database — Required.** Firestore for `users`, `families`, `sessions`, `recipes`, `recipe_steps`, `tips`, `glossary`, `consent_records`, `family_members`. - **File storage — Required.** Firebase Storage for the recorded session audio (preserved at upload sample rate, forever, for the heirloom mode) + per-step photographs + the printable recipe-card PDFs. **Storage is NOT auto-provisioned by AI Studio Build today** — enable it in the Firebase console and wire the bucket name into the AIS Build project before the first session begins recording. Pre-signed URLs only; the audio is never publicly addressable. - **Email — Required (transactional).** Family invitations via magic link (Firebase Auth). Recipe-card share emails with PDF attached. No marketing, no newsletter. - **Payments — Not needed for v1.** Free for personal use. A future "printed family cookbook" tier could pipe to a print-on-demand partner and charge for the physical artefact only. - **External APIs:** Gemini API for all intelligence. Optional measurement-conversion table is bundled locally (no API call); the parenthetical "≈ 60 g" appears via a small built-in unit map covering chhatak, anjili, masha, tola, ratti, fistful, handful, pinch, _puñado_, _cucharada colmada_, _yong cu_ (兩), _gam_ (斤), and the common kitchen units across South Asia, East Asia, Latin America, the Middle East, and Africa. **Environment variables:** every secret (Gemini API key, Firebase service-account JSON) lives in environment variables — never in client bundle. Include a `.env.example`. **Auth + data privacy reminders:** never log secrets · never store passwords in plain text · use HTTPS everywhere · honour 'delete my account' inside the UI · explicit opt-in for any analytics · the grandmother's recorded voice and the family's recipes are never used to train any model. This app uses the Gemini API on the paid tier, where Google does not use your content for model training, per the Gemini API Additional Terms. Recorded audio is encrypted at rest and only ever accessible to the family members the user explicitly invites. **Read this first — prompt-craft rules that apply to every call in this template:** 1. **Name the model variant explicitly** in every Gemini API call. Do not let the agent pick the model. See the per-call matrix below. 2. **Pin `thinkingLevel` explicitly** per call where the model supports it. Live API and TTS do not — see the matrix. 3. **Seed the JSON Schema as a fenced TypeScript / Zod block** in the system instruction or `responseSchema` field. The literal schema is below. **Convert the Zod schema to Gemini's `Schema` type via the SDK helper** before passing to `responseSchema` — do NOT pass raw Zod. **Numeric `min`/`max` constraints are documentation only inside `responseSchema`; clamp on the server after the response arrives.** 4. **Pin the system instruction separately** from user input. Use the `systemInstruction` field for persona + behavioural rules; use `contents` (or the Live API `setup` message) for user input. Never concatenate. 5. **Pre-declare tools as an enable/disable list** per call. The matrix below names which tools are enabled per call. Tools NOT listed for a call should be disabled. 6. **State negative constraints explicitly** — they are listed below. They are NOT "be careful" suggestions; they are hard rules the model must follow. 7. **Live API session config:** set `responseModalities: ["TEXT", "AUDIO"]` (text always, audio toggled by the grandchild). Pin the source-language `inputAudioTranscription` config. Pin the output `speechConfig.voiceConfig.prebuiltVoiceConfig.voiceName` to a warm voice in the target locale. Tune `realtimeInputConfig.automaticActivityDetection.silenceDurationMs` to 4000 (four-second silence threshold — keep her line on the tile while she works). 8. **`responseSchema` and `google_search` are mutually exclusive in one Gemini call.** No call in this template combines them. If you ever add a grounded call (e.g. "what is _chhatak_ in modern grams?"), emit JSON in the body and parse server-side. 9. **Grounded responses can wrap JSON in ```json fences or add prose preamble.** Server-side, strip fences and brace-extract: ```typescript function safeExtractJSON(raw: string): T { const clean = raw.replace(/```json\s*|```/gi, '').trim(); const s = clean.indexOf('{'); const e = clean.lastIndexOf('}'); if (s === -1 || e === -1) throw new Error('No JSON boundaries in grounded response'); return JSON.parse(clean.slice(s, e + 1)) as T; } ``` 10. **Strip unsupported Zod modifiers before passing to `responseSchema`** — Gemini's OpenAPI subset rejects `.regex()` / `pattern`, fixed-length `z.tuple()`, and other custom validators. Use a sanitizer that flattens tuples to length-2 arrays and removes regex patterns before serializing. Validate those constraints in middleware AFTER parsing. ### Per-call model + tools matrix | Call | Model | thinkingLevel | Tools enabled | |------|-------|---------------|---------------| | Live cooking session — speech in, translated text + optional translated audio out | `gemini-3.1-flash-live-preview` | n/a | (none) — bidirectional Live session | | Per-step parse (every ~10 seconds of session transcript → `RecipeStep`) | `gemini-3.5-flash` | low | (none) | | Session-end structured extraction → `Recipe[]` schema | `gemini-3.5-flash` | medium | (none) | | Dialect detection at session start (10-second sample) | `gemini-3.5-flash` | low | (none) | | Photo-step confirmation (photo + step transcript) | `gemini-3.5-flash` | low | (none) | | Recipe-card hero illustration (optional) | `gemini-3.1-flash-image` | n/a | n/a | | Recipe-card heirloom voice clone (opt-in, gated) | `gemini-3.1-flash-tts-preview` | n/a | n/a | *Note for builders:* on TTS and image-generation calls, omit `thinkingConfig` entirely — the field is not supported on those models. The Live API also does not accept `thinkingConfig`. The `n/a` cells in this matrix are documentation only; do not serialise them into the request body. ### Primary structured-output schema (seed this verbatim in the prompt) ```typescript import { z } from "zod"; const Ingredient = z.object({ name_verbatim: z.string(), // "ghee", "chhatak ghee", "pak choi" quantity_verbatim: z.string().nullable(), // "1 chhatak", "a fistful", "un puñado" unit_verbatim: z.string().nullable(), // "chhatak", "fistful", "puñado" modern_conversion_note: z.string().nullable(), // "≈ 60 g" — populated only at first mention inferred_substance_category: z.enum([ "produce", "meat", "fish", "dairy", "grain", "spice", "herb", "fat", "liquid", "condiment", "stock_base", "leavening", "sweetener", "other", ]), contextual_quote_source: z.string(), // verbatim grandmother phrase language_of_name: z.string(), // BCP-47, "yue-Hant-HK", "ta-IN" is_brand_specific: z.boolean(), // true for "Maggi cube" }); const Tip = z.object({ text_source_verbatim: z.string(), // "唔好炒到燶" / "don't let it burn" text_translation: z.string(), // "don't let it burn" language_source: z.string(), // BCP-47 category: z.enum([ "safety", "doneness_cue", "substitution_warning", "family_lore", "personal_preference", "tool_advice", "timing", "sensory_cue", "other", ]), attached_to_step_id: z.string().nullable(), audio_segment_ref: z.string().nullable(), // pointer into session-audio index }); const CodeSwitch = z.object({ phrase_source_verbatim: z.string(), // "appa", "Maggi cube", "ghee" language_inserted: z.string(), // BCP-47 of the inserted phrase language_carrier: z.string(), // BCP-47 of the surrounding sentence translation_or_gloss: z.string(), // "father", "Maggi cube (chicken bouillon)" is_loanword_kept: z.boolean(), // true for "ghee" — kept verbatim in English attached_to_step_id: z.string().nullable(), }); const RecipeStep = z.object({ step_id: z.string(), step_number: z.number().int().min(1), action_source_verbatim: z.string(), // her exact words action_translation: z.string(), ingredients_used_in_this_step: z.array(z.string()),// references Ingredient.name_verbatim time_estimate_seconds: z.number().int().nullable(), temperature_hint: z.string().nullable(), // "medium-high", "until smoking" sensory_cue: z.string().nullable(), // "until the mustard seeds pop" doneness_cue: z.string().nullable(), // "when the prawns turn pink" photo_uri: z.string().nullable(), audio_segment_start_ms: z.number().int().nullable(), audio_segment_end_ms: z.number().int().nullable(), tips_attached: z.array(z.string()), // Tip ids code_switches_in_this_step: z.array(z.string()), // CodeSwitch ids confidence: z.number().min(0).max(1), }); const Recipe = z.object({ recipe_id: z.string(), dish_name_source_verbatim: z.string(), // "薑蔥蒸魚" dish_name_translated: z.string(), // "ginger and spring onion steamed fish" dish_name_romanised: z.string().nullable(), // "gēung chūng jīng yú" cuisine_origin_note: z.string(), // "Guangzhou-style steamed fish" serves_estimate: z.string().nullable(), // "about 4, as part of a shared meal" ingredients: z.array(Ingredient), steps: z.array(RecipeStep), tips: z.array(Tip), code_switches: z.array(CodeSwitch), source_language_primary: z.string(), // BCP-47 source_dialect_detected: z.string().nullable(), // "Cantonese, Guangzhou register" target_language: z.string(), // BCP-47, usually "en" grandmother_voice_id: z.string().nullable(), // pointer to family voice profile session_audio_uri: z.string().nullable(), session_started_at: z.string(), // ISO session_duration_seconds: z.number().int(), flagged_for_user_review: z.array(z.object({ field_path: z.string(), reason: z.string(), })), lineage_note: z.string().nullable(), // "this is how paati's own paati made it" }); type Recipe = z.infer; ``` ### Common failure modes (and how to avoid them) - Agent silently downgrades `thinkingLevel` on the the session-end structured extraction call to save quota — pin `gemini-3.5-flash` with the matrix-specified `thinkingLevel` explicitly. Flash conflates dish boundaries when paati cooks three things back-to-back; it also drops code-switches. - App auto-translates the dish name and discards the source. Hard rule: `dish_name_source_verbatim` is always preserved; the translated form sits beside it, never replaces it. - The model "helpfully" converts _1 chhatak ghee_ into _4 tablespoons ghee_ and drops the source — wrong. The system instruction requires the verbatim quantity and unit; the modern conversion is appended in `modern_conversion_note` on first appearance only. - Cantonese transcribed as Mandarin pinyin — the Live session's `inputAudioTranscription.languageCode` must be pinned to the detected dialect (e.g. `yue-Hant-HK`, not `zh-CN`). Pin it during dialect-detection step at session start, not lazily. - Maghrebi Arabic flattened to Modern Standard Arabic — same fix: pin the dialect at session start; reject any structured-extraction output where `source_dialect_detected` reverts to a generic top-level locale. - The Live API is configured with `responseModalities: ["AUDIO"]` and the text tile goes blank — always include `"TEXT"`. The text tile is the load-bearing UI; audio is optional. - Code-switches dropped. Tamil-English switching in particular: paati says "_appa told me pressure cooker la podu_" and the model outputs "_father told me to put it in the pressure cooker_" without preserving the bilingual structure. The schema requires `code_switches[]`; the system instruction requires populating it on every detected switch. - The TTS reads the grandchild's tap-to-pause as a continuation. Always check the local mute flag before piping text into the TTS output queue. - Per-step parse runs every word — too noisy. Buffer the Live transcript and call the per-step extraction every ~10 seconds of speech or at every detected long pause, whichever is first. - Photo confirmation overrules the grandmother. Hard rule: if the photo-confirmation call says "this does not look like _crisp brown crust_", the chip is advisory and says "paati said _crisp brown crust_; the photo looks pale — keep going?" — it never says "she was wrong". - Voice clone enabled by default. NEVER. Voice clone is opt-in, requires a spoken consent recording from the grandmother, requires the family-account owner to actively enable it, and is never used for any purpose outside this family's archive. - Live session timeout in mid-cook because the kid forgot to interact with the screen. Set the screen wake-lock when the session starts; do not let mobile browsers sleep the tab while the Live API is active. ### Negative constraints (hard rules) - Do NOT substitute ingredients in translation. _Ghee_ stays _ghee_. _Pak choi_ stays _pak choi_. _Maggi cube_ stays _Maggi cube_. _Banaba leaves_ stay _banaba leaves_. No "Western equivalents". No "or use butter". The recipe is the grandmother's recipe. - Do NOT translate proper nouns. First names ("Wai Po", "Paati", "Lola", "Yiayia", "Abuela", "Nana", "Nai-nai", "Tatang"), dish names ("_kuzhambu_", "_pho_", "_pancit_", "_pupusa_", "_baklava_", "_doro wat_", "_nasi lemak_"), brand names ("Maggi", "Knorr", "Aji-no-moto", "Patak's"), and place names stay verbatim. Translation appears as a parenthetical gloss on first occurrence only and never thereafter. - Do NOT modernise units of measurement. _Chhatak_, _tola_, _masha_, _ratti_, _anjili_, _yong_, _gan_, _liang_, _jin_, _puñado_, _cucharada colmada_, _pizca_, "_a fistful_", "_a small handful_", "_the spoon I use for the tea_" all stay verbatim. Modern conversions are appended on first occurrence only in `modern_conversion_note`. - Do NOT auto-correct dialect to "standard". Cantonese is not Mandarin. Cebuano is not Tagalog. Maghrebi Arabic is not Levantine. Calabrian is not Standard Italian. Quechua-inflected Spanish is not Castilian. Detect, pin, respect. - Do NOT flatten code-switching. When she switches languages mid-sentence, both layers are preserved. The translation shows the dominant language; the inserted phrase appears in its original beside the translation. - Do NOT "smooth" the grandmother's voice. _"Don't let the curry leaves burn, you know how it smells when they burn"_ does NOT become _"Avoid over-toasting the curry leaves"_. The English is a translation of her — not a chef-school rewrite. - Do NOT use family recordings to train or fine-tune any model. Use the Gemini API on the paid tier, where Google does not use your content for model training, per the Gemini API Additional Terms. - Do NOT auto-publish or auto-share recipes. Every recipe is private to the family; sharing is explicit, per-recipe, per-relative. - Do NOT clone voices without explicit spoken consent and family-owner opt-in. Voice clone is a deliberate, late-stage heirloom feature, not a default. - Do NOT cut her off when she pauses. Four seconds of silence is "she is concentrating on the knife"; thirty seconds of silence is "she is plating". Neither triggers a "session over" event. - Do NOT interrupt with clarifying questions during the Live session. If the model is uncertain, surface a small chip on the recipe card after she has finished the step — never as a spoken interruption. - Do NOT show ads, dish suggestions from other families, or "related recipes". This app is just her dish. ### Per-call `systemInstruction` strings Use these as the literal `systemInstruction` field for each Gemini API call the built app makes. They complement the series-wide rules already uploaded as the global instructions file (`00-series-instructions.txt`). ### Call: Live cooking session — speech in, translated text + optional translated audio out Model: `gemini-3.1-flash-live-preview` · thinkingLevel: n/a · Tools: (none) ``` You are translating, in real time, a grandmother teaching her grandchildren to cook. She is speaking in the source dialect the client has pinned in this session's configuration. The grandchildren do not speak her language. They are reading the target-language tile on the counter and, when they have enabled audio, listening to a calm voice in the target language. Languages and dialects you will encounter include: Cantonese (Guangzhou, Hong Kong, Toishan, Hoiping), Mandarin (Beijing, Shanghai, Taiwanese), Hokkien, Hakka, Vietnamese (Northern, Central, Southern), Tagalog, Cebuano, Hiligaynon, Ilocano, Kapampangan, Korean, Japanese, Thai, Lao, Khmer, Burmese, Tamil (Chennai, Madurai, Jaffna, Malaysian), Telugu, Malayalam, Kannada, Hindi, Urdu, Punjabi (Gurmukhi or Shahmukhi), Bengali, Sinhala, Marathi, Gujarati, Pashto, Dari, Farsi, Turkish, Kurdish (Kurmanji or Sorani), Armenian, Greek, Arabic (Levantine, Egyptian, Maghrebi, Gulf, Yemeni, Iraqi), Hebrew, Yiddish, Amharic, Tigrinya, Oromo, Swahili, Yoruba, Igbo, Twi, Wolof, Hausa, Spanish (Salvadoran, Mexican, Cuban, Andean, Castilian, Argentine, Chilean, Caribbean), Portuguese (Brazilian, Continental, Cape Verdean), Italian (Standard, Sicilian, Calabrian, Neapolitan), Polish, Ukrainian, Russian, Romanian, Quechua, K'iche', Haitian Creole. Detect dialect markers and pin to the most specific code your inventory supports. Your job in this session: 1. Listen continuously to the grandmother's speech. 2. Stream the translated text into the target-language output as soon as a meaningful clause has completed. Do NOT translate word-by-word; do NOT wait for whole paragraphs. 3. When the target-language audio modality is enabled, speak the translated text in a warm, unhurried voice paced for cooking (~110 words per minute). Do NOT overlap with her ongoing speech; wait for her to finish a clause before speaking. 4. When she pauses for less than four seconds, do NOT speak. Hold the last line on the text tile. She is thinking, slicing, or stirring. Silence is not your cue. 5. When she pauses for more than thirty seconds, do NOT end the session. Hold the tile and wait. If the family wanted to end, they would press the button. 6. If she code-switches into another language (e.g. Tamil into English for "pressure cooker", Cantonese into Mandarin for a dish name, Spanish into K'iche' for an ingredient), translate the dominant language as the main output and surface the inserted phrase as a brief side note in the text tile, formatted as: "_phrase_verbatim_ · source-language → translation". Do NOT collapse the switch silently. Hard rules: - Do NOT translate proper nouns: first names, dish names ("kuzhambu", "pho", "pupusa"), brand names ("Maggi cube", "Patak's"), place names. - Do NOT translate units of measurement. "1 chhatak ghee" stays "1 chhatak ghee". A parenthetical conversion (e.g. "≈ 60 g") may be appended on first mention by the downstream structured pass, NOT by you in real time. - Do NOT substitute ingredients. "Pak choi" is "pak choi", not "Chinese cabbage". "Ghee" is "ghee", not "clarified butter". - Do NOT smooth her voice. Translate her, not what a chef-school textbook would say. If she says "don't let the curry leaves burn, you know how it smells when they burn", the English is "don't let the curry leaves burn, you know how it smells when they burn". - Do NOT interrupt with clarifying questions. If you are uncertain about a word, render your best reading and flag it for the downstream structured pass via your transcription confidence signal. - Do NOT auto-end the session. Only end when the client signals end-of-session. - Do NOT speak when the audio modality is muted by the user. - Do NOT use SSML phoneme tags or break-time tags. Pacing is via ellipsis at line ends, blank-line plus em-dash at paragraph ends, and the style directive you have been given. - Preserve every diacritic exactly. Polish ł ą ę ó ż ź ć ń ś; Vietnamese tone marks; Pinyin tone marks when they appear in romanisation; Portuguese ã õ ç; Spanish ñ ü; Arabic and Hebrew full vocalisation when she pronounces it. If the grandchild taps "hand back" mid-session, stop translating output until they tap "resume". She is telling a side-story; let her tell it without recording it onto the recipe card. No commentary. Just translate. ``` --- ### Call: Per-step parse (every ~10 seconds of session transcript → `RecipeStep`) Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: (none) ``` You receive a rolling window of the most recent ~10 seconds of the Live session transcript (source language + target language) plus the current open `RecipeStep` (if any). Your job: decide whether the last 10 seconds extends the current step, ends the current step, starts a new step, or is conversation that does not belong to any step. Hard rules: - An action verb in the source language ("chop", "stir", "fry", "let it rest", "add the salt") usually marks a new step. Two consecutive sentences with the same action ("now stir", "keep stirring") usually mark a continuation, not a new step. - A dish-name announcement ("now I will make the soup") marks a new Recipe boundary, not a new step within the current recipe. - Side-conversation (the grandmother asking the grandchild about university; gossip about a cousin) is NOT a step and is NOT a tip. Output `"type": "off_topic"` and do not pollute the recipe. - A safety warning, doneness cue, or tip ("be careful, the oil splashes here"; "until the mustard seeds pop") is a Tip, not a step. Attach it to the current step if one is open. - Preserve `action_source_verbatim` exactly. Do not paraphrase. - Estimate `time_estimate_seconds` only from explicit cues she said ("for about two minutes", "until the rice is done"). Otherwise null. - Preserve `temperature_hint` verbatim from her ("medium-high", "low", "until smoking", "lower than that"). - Identify ingredients used in this step by name — do NOT invent ingredients she did not mention; reference only names already in the Recipe.ingredients[] array maintained by the session. Output: a single JSON object with one of these shapes: { "type": "continue_step", "step_update": { ... }} { "type": "end_step", "step_final": { ...RecipeStep }} { "type": "new_step", "step": { ...RecipeStep }} { "type": "new_recipe_boundary", "next_dish_name_verbatim": "..." } { "type": "tip", "tip": { ...Tip }} { "type": "off_topic" } { "type": "code_switch_only", "code_switch": { ...CodeSwitch }} No commentary outside the JSON. ``` --- ### Call: Session-end structured extraction → `Recipe[]` schema Model: `gemini-3.5-flash` · thinkingLevel: medium · Tools: (none, long context) ``` You receive the full session transcript: every spoken sentence the grandmother said (source language), the live translations that were produced in the moment (target language), and the per-step decisions the streaming parser made. Your job: produce one or more complete `Recipe` objects. One session may contain one dish or three; honour the boundaries the streaming parser detected, and only override them if there is clear evidence (e.g. she explicitly said "now a different dish"). Hard rules: - `dish_name_source_verbatim` is the grandmother's exact name for the dish, in her language and script. Do NOT romanise unless she romanised. Romanisation goes in `dish_name_romanised`. - `dish_name_translated` is the natural-language English (or target-language) name a reader would recognise — but it must not erase the source name; both fields coexist. - `cuisine_origin_note` is a single short phrase identifying the cuisine and, where she said it, the regional variant ("Guangzhou-style steamed fish"; "Chettinad chicken kuzhambu"; "Salvadoran cheese pupusa"). Do NOT make up regional origins she did not state. - `serves_estimate` only if she said it ("enough for 4 of us", "for the table"). Otherwise null. - `ingredients[]` — every ingredient she mentioned, with `quantity_verbatim` and `unit_verbatim` preserved exactly. Do NOT merge or split; do NOT modernise units. Populate `modern_conversion_note` ONLY on the first appearance of a non-metric/non-volumetric unit she used (chhatak, anjili, tola, liang, gan, jin, fistful, puñado, cucharada colmada, ratti). Use the bundled conversion table; if a unit is not in the table, set `modern_conversion_note` to null and flag for review. - `is_brand_specific: true` for any branded ingredient she named ("Maggi cube", "Knorr", "Patak's", "Aji-no-moto"). Do not strip the brand. - `steps[]` — every step in the order she described. Action verbatim in `action_source_verbatim`; clean English translation in `action_translation`. Preserve `sensory_cue`, `doneness_cue`, `temperature_hint` verbatim from her. - `tips[]` — every aside, warning, family-lore line, and doneness cue. Verbatim source; translated; tagged with category. - `code_switches[]` — every detected language switch. Preserve the inserted phrase, the carrier language, the inserted language, the translation, and whether the phrase is a loanword that survives untranslated in English ("ghee", "pak choi", "tortilla"). - `source_dialect_detected` is the dialect pinned at session start. Do NOT collapse to a generic top-level locale. - `flagged_for_user_review[]` — name any field where confidence is below 0.7 with a one-sentence reason. - `lineage_note` only if she said so ("this is how my paati made it", "I changed it a little when I came to London"). Output ONLY the array of Recipe JSON matching the provided schema. No commentary. ``` --- ### Call: Dialect detection at session start Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: (none) ``` You receive the first ~10 seconds of audio from a cooking session. Identify the language and the most specific dialect. Output: { "language_bcp47": "yue-Hant-HK", "dialect_descriptor_human": "Cantonese, Hong Kong register", "confidence": 0.92, "alternates_considered": [ { "language_bcp47": "yue-Hant-CN", "dialect_descriptor_human": "Cantonese, Guangzhou register", "confidence": 0.71 } ] } Hard rules: - Prefer the most specific dialect code your inventory supports. Do NOT collapse Cantonese to `zh-CN`; do NOT collapse Maghrebi Arabic to `ar`. - If she greets the kids by a kinship term ("nai-nai", "paati", "lola", "abuela"), use that as a strong signal for the dialect cluster and region. - If confidence is below 0.6, ask the user (via the client) to pick from a short menu of the top 3 candidates. No commentary outside the JSON. ``` --- ### Call: Photo-step confirmation Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: (none) ``` You receive: one photograph of the pan/pot/plate at the moment the grandchild tapped capture, plus the grandmother's verbatim description of the current step (source language + translation). Your job: in one short sentence, say whether the photo is broadly consistent with what she just described. Hard rules: - Advisory only. NEVER assert she was wrong. - If the photo shows something materially different from her described state ("she said crisp brown crust; the photo looks pale"), surface it as a soft chip the grandchild can choose to show her or not. - Vocabulary stays domestic, not chef-school. "Looks pale" not "lacks Maillard development". - If the photo is too blurry or dark to judge, say so plainly. - Do NOT recommend changes to her recipe. Output: { "consistency": "consistent" | "advisory_mismatch" | "image_unclear", "note_for_grandchild": "string — one short sentence", "should_surface_as_chip": boolean } No commentary outside the JSON. ``` --- ### Call: Recipe-card hero illustration Model: `gemini-3.1-flash-image` · n/a · n/a ``` Generate a single, warm, watercolour-leaning illustration for the top of the printable recipe card. Style direction: hand-painted watercolour on warm cream paper. Loose brushwork. Soft shadows. No glossy "AI render" look. No people. No text in the image. Centred on the dish or its key ingredient. Restrained palette: 4-5 muted tones drawn from the dish's ingredients (the green of curry leaves; the cream of coconut; the brown of ghee; the red of dried chilli). The dish is: {{dish_name_source_verbatim}} ({{dish_name_translated}}). Key ingredients: {{ingredients_top_3}}. Hard rules: - No human figures. - No text or lettering in the illustration. - No utensils unless they are central to the dish (e.g. a clay pot for clay-pot rice). - Square 1:1 aspect ratio. - Warm cream paper background, not white. Output: a single image. ``` --- ### Call: Recipe-card heirloom voice clone (opt-in, gated) Model: `gemini-3.1-flash-tts-preview` · n/a · n/a ``` Voice: the cloned grandmother voice the family has explicitly enrolled with spoken consent. If no clone is available, fall back to the nearest warm voice for `target_language` from the Gemini 2.5 Flash TTS catalogue. Read the recipe-card text in the target language at a warm, unhurried pace. Pre-process the text before sending it to TTS: - Read the dish name first, source-verbatim then translated, e.g. "薑蔥蒸魚 · ginger and spring onion steamed fish". The TTS will pronounce the source name using the voice's native locale; the family knows their grandmother's voice will not perfectly read characters she does not normally speak aloud, and that is fine. - Insert a single ellipsis (`…`) at every line break. - Insert a blank line plus an em-dash (`—`) at every step boundary. - Target rate: ~110 words per minute — kitchen-reading pace. Style direction: prepend ONE short directive sentence to the input text, exactly like: "Read warmly and unhurriedly, as if reading a family recipe aloud to a grandchild at the kitchen table. …". Gemini 2.5 TTS has no separate `style` API field; the directive sentence is how style is conveyed. Pronunciation: no SSML phoneme overrides — Gemini 2.5 TTS does not support `` or `` tags. Pronunciation comes from the chosen voice's native locale. The grandmother's own loanwords ("ghee", "pak choi") will be read with her natural accent if her voice is the active clone. No commentary outside the audio. ``` ## 5. Use cases & content to include Build dedicated UI sections or flows for each of these — they tell you what content the app must support. - **The Saturday-afternoon class.** A Sydney-born brother and sister, ages 9 and 12, set up the phone on the counter while their nai-nai, visiting from Guangzhou for six weeks, decides to teach them her ginger-and-spring-onion steamed fish. She speaks Guangzhou-register Cantonese. They speak English. The session runs 47 minutes; she teaches one dish, with two long pauses (one to debone, one to wash spring onions), and ends by telling them which fishmonger her mother used to use. - **The three-dish afternoon.** A Tamil-Brit grandson sets up the app on his counter when his paati arrives from Chennai. She announces in Tamil that she will cook three things: rice, fish kuzhambu, and a small thoran. The app auto-detects the dish boundaries and produces three separate recipe cards by the end. Half her sentences slip into English for kitchen nouns ("pressure cooker", "Maggi cube"); each switch is preserved as a code-switch chip beside the translated step. - **The pupusa lesson.** A Salvadoran-American granddaughter cooks with her abuela in San Francisco's Mission District. The abuela speaks fast Salvadoran Spanish with K'iche' words for corn ingredients she learned from her own grandmother. Two K'iche' inserts are detected, preserved, and labelled. The granddaughter learns that the word her abuela uses for masa, _xkik'_, is not a Spanish word at all. - **The 30-second pause.** Mid-step, the grandmother stops talking. She is concentrating on slicing ginger. The app does not start prompting. The last line ("now slice the ginger as thin as paper, then a little thinner") sits on the tile for thirty-one seconds while she works. When she speaks again ("now put it in the bowl"), the next line appears and the previous one slides up. - **The "hand back" gesture.** Mid-cook, the grandmother starts telling a story about her own mother's kitchen in pre-war Guangzhou. The grandchild taps "hand back" — the app keeps listening but stops translating to the tile. The story is recorded into the heirloom audio archive but does not appear on the recipe card. When she returns to the dish ("OK, now the soy sauce"), the grandchild taps "resume" and the translation continues. - **The fistful.** The grandmother says "_a fistful of dried shrimp_". The app does NOT convert this to "¼ cup of dried shrimp". The ingredient on the recipe card reads "a fistful of dried shrimp", with no conversion appended — because there is no honest conversion for "her fist". - **The brand-name rule.** A Filipina lola says "_isang piraso ng Maggi cube_". The English tile reads "one Maggi cube". The recipe card lists "Maggi cube" as a branded ingredient. The card does NOT helpfully substitute "stock cube". - **The pressure-cooker switch.** A North-Indian dadi says "_phir pressure cooker mein daal do_". The English tile reads "then put it in the pressure cooker". A code-switch chip below labels "_pressure cooker_ · English loanword". The recipe card preserves both layers. - **The dialect surprise.** The dialect-detection step at session start identifies a paati from Jaffna, not Chennai. The chip on the tile shows "_Tamil, Jaffna register_". The grandchild texts her father in the next room: "_Dad — did paati grow up in Jaffna?_". He says yes, and explains for the first time how the family moved. The app surfaces a moment that was hiding in plain sight. - **The chhatak conversion.** A Bengali dida says "_ek chhatak ghee_". The first appearance in the recipe card reads "1 chhatak ghee (≈ 60 g)". When she uses chhatak again later, the card reads "2 chhatak ghee" — no second conversion, because the family has already learned what a chhatak is. - **The three-cities share.** After the session ends, the grandchild taps "share with cousins" and adds her cousin in Singapore and her aunt in Auckland. They each receive the recipe card plus a link to play the grandmother's voice on any step. Two weeks later her cousin in Singapore adds a note: "_paati used to do it with curry leaves instead — she changed when she moved._" - **The heirloom save.** The granddaughter, six months later, opens the app on a Wednesday night and presses play on step 4 of the kuzhambu. Her paati's voice, recorded on a Saturday afternoon in May, says "_now the curry leaves — one second only, don't let them brown_". Paati died two months ago. The recipe still works. - **The voice clone opt-in.** A year after the heirloom save, the granddaughter is teaching her own three-year-old the dish. With the family-account owner's deliberate opt-in, the cloned voice reads the recipe card aloud — a small heirloom moment for the next generation. The clone is never used outside the family's own archive; it is never sent for training. ## 6. Page structure Build the following screens / sections in this order. Adjust copy to fit the voice, but keep the structural intent. 1. **Welcome / sign-in.** A photographed-looking image of a phone propped on a wooden kitchen counter beside a chopping board with a few ginger pieces and spring onions. Warm late-afternoon light. One paragraph: "Kitchen Phrasebook lets a grandmother teach her grandchildren to cook in any language. She speaks. They cook." Single Google sign-in button; Apple sign-in next to it. Below: "Try with the sample session" → loads the demo session in section 8a. 2. **Empty state — "Set up the kitchen".** Two big input methods: 🎙 Start a cooking session · 📝 Add a pre-recorded voice memo. A short explainer below each ("Best when grandma is here with you in the kitchen", "Best for the dish a relative recorded for you years ago"). A third, smaller link: "Browse our family's saved recipes". 3. **Language picker** (first launch only; remembered after). Top of the screen: "Who is teaching today?" Below: a long, searchable, dialect-granular language list. Recently-used languages stick at the top after first use. Helper text: "We try to identify her dialect once she starts speaking. You can override here if we get it wrong." 4. **Live session — Counter Mode** (the hero screen). Full-screen single tile. The current translated line is the largest type on the screen (≥ 36 px, scales up on tablet to ≥ 64 px). Above the line: the dish name detected so far + the current step number + a tiny dialect chip ("_Cantonese, Hong Kong register_"). Below the line: the previous two lines, smaller and dimmed. To the side: a "history" two-finger swipe scrolls through everything so far. Floating controls on the lower edge: 🔊 toggle audio (English voice on/off) · 📸 capture step photo · ✋ hand back (pause translation but keep recording) · ⏹ end session. The screen wake-lock is on while the session runs. 5. **Mid-session recipe card preview (collapsible).** A drawer accessible by a small chevron on the side of the Counter Mode tile. Slides out the recipe card as it is being built: ingredients top, steps listed, tips below. Updates live every ~10 seconds. Tapping a step jumps the audio scrubber to that moment. 6. **Session end — review screen.** The full recipe card laid out at reading size. Dish name in both languages, romanisation underneath. Ingredients column with quantities and units verbatim, conversion notes parenthetically on first occurrence. Steps numbered. Tips in their own warm-tinted panel ("paati says…"). Code-switches in a small "her words" panel. A button: "Save to family archive". A second button: "Edit before saving" — opens an inline editor for any field; everything is editable, nothing is auto-locked. 7. **Family archive view.** Magazine-style grid of saved recipe cards. Filter by cook (paati / lola / abuela / nai-nai / dad), cuisine, language, dish type. Default sort: most recently cooked. Each card shows the dish in both languages, the date, the cook, a small play-button to jump to the audio. 8. **Recipe detail view.** The full printable card, plus a vertical "voice scrubber" along the right edge — each step is a vertical bar; tap a bar and the grandmother's voice plays from that step. A small "play the whole session" button. A "print" button (A4 / US Letter / 4×6 index card). A "share with cousins" button (magic-link email). 9. **Family members.** Who is in this family archive. Add a cousin in another city; assign each member their relationship label (cousin, aunt, sister, mum); see who has added what. No global social graph. No public profiles. 10. **Cook profiles.** One screen per cook (paati, nai-nai, lola, abuela). Their dishes, their dialect, the total minutes of their voice the family has preserved, the date of the first session, the date of the most recent. A consent record: when she gave spoken consent and what she consented to. 11. **Voice clone opt-in (gated, late-stage).** Reached only by the family-owner from cook-profile settings, only once a cook has at least 60 minutes of recorded audio across multiple sessions. A clear explanation of what cloning means, what the clone will and will not be used for (only for re-reading her own saved recipes inside this family's archive; never for training; never for sharing outside the family), and a spoken-consent recording flow. The cook records: "_I am [name]. I agree that my grandchild [name] can use my voice to read my recipes aloud in this app for our family only._" Audio is preserved as the consent record. The clone is never used until this step is complete. 12. **Glossary.** A small editable per-family list of preferred names ("she calls it _ammini_, not _ammuli_"). The model uses these in the next session. 13. **Settings.** Language defaults, audio voice for the target language, default dialect overrides, family invitations, delete-this-archive-forever. Capabilities `(i)` icon in header. 14. **Footer.** "Made for the recipes nobody else can read." Privacy: "Your family is yours. We never train on it." Capabilities info button in header. ## 6b. First-visit onboarding Show a **first-visit onboarding** the first time a visitor lands on the app (detect via `localStorage` flag; do not show on return visits). Three slides, dismissible at any time. Persistent re-entry: a `?` icon in the header reopens it. **Slide 1 — What this is.** - Headline: "Welcome to Kitchen Phrasebook." - Subhead: "Your grandmother speaks. You cook. We translate, live, paced so she doesn't have to repeat herself." - One paragraph (≤ 60 words) explaining who this is for and what makes it different from Google Translate's conversation mode: it understands cooking pace (a 30-second silence is not "session over", it is "she is concentrating on the knife"), it preserves her ingredients exactly (1 chhatak ghee stays 1 chhatak ghee), and it writes the recipe card while she cooks. - Visual: a small annotated illustration of a phone on a counter with the labelled regions (live translated line, dialect chip, current step, history swipe, hand-back button) — not a generic phone icon. **Slide 2 — Try it now.** - One short prompt: "Try with the sample session". - A live demo input pre-loaded with the 47-minute Cantonese-grandmother session described in section 8a. - 1-2 sentences pointing at *the specific elements* where the Gemini magic happens: the live translated line, the dialect chip ("_Cantonese, Hong Kong register_"), the preserved code-switches ("_Maggi cube_ · English loanword"), and the recipe card being built in the side drawer. **Slide 3 — How to remix this.** - Headline: "Make this yours." - Three short bullets: - "Swap the sample session in `/data/seed-session/` for your own family's recording, or open the app with your grandparent in the kitchen." - "Adjust the dialect inventory in `/server/prompts/dialect-detection.txt` to fit the dialect cluster your family speaks." - "Wire up your Gemini API key and Firebase project via the env-var list in the capabilities panel." - Primary CTA: "Use this template" → links to AI Studio Build remix entry point. - Secondary: "Just exploring — close" (sets localStorage flag, never auto-shows again). **Accessibility:** focus trap, `Esc` closes, `role="dialog"`, `aria-modal="true"`, `aria-labelledby`, focus restored to trigger on close. Respect `prefers-reduced-motion`. **Don't:** - Don't gate content behind the modal. The page beneath must be fully usable. - Don't auto-reshow on return visits. Use `localStorage['onboarding-seen-v1']`. - Don't include unrelated CTAs (newsletter signup, social follow). Keep it about the template only. ## 6c. Capabilities info button (persistent in header) Add a persistent `(i)` icon in the top-right of the header (next to the primary nav). Click → opens a modal/panel titled **"What powers this app"**. **Panel contents (in this order):** **Gemini capabilities used (the hero list):** - **Gemini Live API (`gemini-3.1-flash-live-preview`)** — bidirectional streaming session. The grandmother's audio streams in; translated text streams out to the counter tile within ≤ 1.2 s. Optional translated audio out, paced for cooking (~110 WPM). Voice-activity detection tuned to the kitchen — a four-second pause is normal, a thirty-second pause is normal, nothing auto-ends the session. - **Gemini 3.5 Flash (structured output)** — at session end and at every detected dish boundary, the rolling transcript is passed to a structured-extraction call that produces the typed `Recipe` JSON: ingredients with verbatim quantities, steps in order, tips, and detected code-switches. This is the call that produces the printable recipe card. - **Gemini 3.5 Flash (per-step parse + dialect detection)** — runs every ~10 seconds during the Live session to decide whether the last clause extends the current step, starts a new step, is a tip, or is off-topic conversation. Also runs at session start on a 10-second audio sample to identify the dialect. - **Gemini 3.5 Flash (multilingual)** — translates Cantonese, Mandarin, Hokkien, Tagalog, Cebuano, Vietnamese, Korean, Tamil, Telugu, Malayalam, Hindi, Urdu, Bengali, Punjabi, Sinhala, Arabic dialects, Farsi, Amharic, Swahili, Yoruba, Spanish dialects, Portuguese dialects, Italian regional dialects, Polish, Russian, Greek, Quechua, K'iche', and many more, with dialect awareness. - **Gemini 3.5 Flash (multimodal)** — at every step photo capture, compares the photo with the grandmother's verbatim description and surfaces a soft chip only if they materially differ ("paati said _crisp brown crust_; the photo looks pale — keep going?"). Advisory only, never overriding. - **Gemini 2.5 Flash Preview TTS** — speaks the translated text in a warm, unhurried voice at ~110 WPM when the grandchild enables English audio. Also used for the optional, opt-in, consent-gated voice-clone heirloom mode. - **Nano Banana 2 (`gemini-3.1-flash-image`)** — generates a warm, watercolour-leaning hero illustration for the printable recipe card. Used sparingly; never replaces the photographs the grandchild took during the session. - **Firebase Auth** — Google and Apple sign-in, family invitations via magic links. - **Firestore** — stores recipes, sessions, family members, glossary, consent records. - **Firebase Storage** — keeps the original session audio at upload sample rate, forever, for the heirloom mode. **Backend services this app depends on:** - Auth: see section 4b - Database: see section 4b - Storage: see section 4b - Email: see section 4b - Payments: see section 4b (not used in v1) - External APIs: see section 4b **Environment variables you'll need to configure:** - `GEMINI_API_KEY` — your Google AI Studio API key - `FIREBASE_PROJECT_ID` — your Firebase project id - `FIREBASE_SERVICE_ACCOUNT` — service-account JSON (server-side only) **Cost + privacy notes:** - One short paragraph per cost-sensitive capability: Live sessions are billed per second of audio in and tokens of text/audio out — a typical 45-minute session costs about $0.90, total. - One short paragraph on privacy: where the data lives (your Firebase project), how to delete it (Settings → "Delete this archive forever" — gone in 60 seconds), and what is never sent for training. The grandmother's recorded voice and the family's recipes are private to the family members the owner explicitly invites; nothing leaves the project. **Documentation links:** - AI Studio Build docs - Gemini Live API docs (session config, voice activity detection, response modalities) - Gemini API multimodal, multilingual, long-context, TTS docs - Firebase Auth, Firestore, Firebase Storage docs **Accessibility:** same standards as the onboarding modal — focus trap, `Esc`, ARIA, restored focus. **Behaviour:** - Always available — single click from anywhere in the app. - Tooltip on the `(i)` icon: "How this app is built". - Mobile: opens as a full-screen sheet that slides up. - Should be the most honest part of the app — never hand-wave service requirements; never say "AI" without naming the specific Gemini model and capability. ## 6d. Detailed cost breakdown (deployer reads this BEFORE shipping) - **Live session (Gemini Live `gemini-3.1-flash-live-preview`)** — billed per second of audio input + per token of text-and-audio output. A 45-minute session with continuous audio in, ~6,500 tokens of translated text out, and ~30 minutes of translated audio out (audio off for ~⅓ of the session, as the grandchild typically reads) ≈ ~$0.90/session. A short 10-minute session ≈ ~$0.20. - **Dialect detection (Gemini 3.5 Flash, low thinking, 10-second audio sample)** — ~$0.0008/session. Runs once per session. - **Per-step parse (Gemini 3.5 Flash, low thinking, every ~10 s of session)** — ~270 calls in a 45-minute session, each call ~300 input tokens + ~80 output tokens ≈ ~$0.012 per session. - **Session-end structured extraction (Gemini 3.5 Flash, medium thinking, long-context over full session)** — typical 45-minute session ≈ ~12,000 input tokens + ~3,500 output tokens. ~$0.03/session. - **Photo-step confirmation (Gemini 3.5 Flash, low thinking)** — ~$0.006 per photo. Typical session has 4-8 step photos. ~$0.04/session. - **Recipe-card hero illustration (Nano Banana 2)** — ~$0.03/image. One image per recipe; cached. - **TTS narration (Gemini 2.5 Flash TTS)** — billed per output token (~$10/M output tokens), effectively ~$0.000003/character. A printable recipe card read aloud ≈ ~1,800 characters ≈ ~$0.005. Cached per recipe. - **Voice clone heirloom playback (Gemini 2.5 Flash TTS, gated)** — same per-character cost. - **Expected per-session cost on first run:** ~$1.00 for a typical 45-minute three-dish afternoon. ~$0.25 for a single 10-minute dish. - **Audio storage:** Firebase Storage standard tier, ~$0.026/GB/month. A 45-minute session at 48 kHz mono = ~165 MB ≈ ~$0.004/month per session. A family's archive of 50 sessions ≈ ~8 GB ≈ ~$0.21/month. - **Bandwidth:** Live API bidirectional audio over the session — typical 45-minute session ≈ ~50 MB downstream + ~50 MB upstream. On home Wi-Fi this is negligible; on a flaky cellular link the offline-fallback transcript catches up after reconnect. ## 7. Design language - **Mood:** A grandmother's kitchen at 4 in the afternoon. Steam on the window. A wooden chopping board. Late light coming sideways across the counter. Not a recipe app. Not a SaaS dashboard. Not a stylised "Asian aesthetic" or "Latin American aesthetic". The chip on the screen says her dialect; the rest of the design recedes so she is the centre of the room. - **Typography:** A warm humanist sans for the live counter tile (Inter, Geist, or DM Sans), set at clamp(28 px → 64 px) so it reads across the counter without anyone leaning in. A display serif (Source Serif Pro or Adobe Caslon Pro) for the printed recipe card and the dish names. A handwriting-styled accent (Caveat, Homemade Apple, or similar) only for the grandmother's tips ("_paati says…_") in the recipe card — never for the live tile, which must stay legible. - **Palette:** Warm cream `#F6EEDC` for the counter-tile background. Deep ink `#1B1714` for body text. Burnt-orange accent `#B85A1E` for the dialect chip and step numbers. Soft sage `#7A8D6E` for tips and code-switch chips. A muted blue `#3A5773` for grandchildren's notes added later (so they do not look like her words). No saturated brand-color buttons. No neon. No gradients. - **Imagery:** The photos the grandchild captured during the session are the canonical images. They sit in the recipe card at warm-rendered quality. The optional Nano Banana 2 hero illustration is small, painterly, and never replaces a photo. Period or ethnic-stereotype imagery is forbidden in app chrome — no claypots-and-spices flat-icon clichés, no "_authentic_" wood textures branded as ethnic-X. - **Hand-feel touches:** A barely-visible paper-grain on the recipe card background. The audio scrubber on the recipe-detail view has a soft analogue feel — the step bars rise and fall as her voice plays, like an old tape level meter. The dialect chip has a tiny micro-animation when first detected: it appears with a single 200 ms fade-in, never with a bounce. - **Spacing:** consistent 4-px base. Counter mode is intentionally airy — the translated line breathes. - **Radius:** consistent token set (6 / 12 / 20 px). Tile uses 12; recipe cards use 6; the welcome card uses 20. - **Shadows:** subtle, layered, warm-tinted. Avoid heavy drop-shadows. The recipe card has a single thin shadow as though resting on the counter. - **Motion:** purposeful — entrance fades, hover lifts, the dialect chip's 200 ms fade-in. The Counter Mode tile's previous-line transition is a 250 ms vertical slide; the current line appears instantly. Respect `prefers-reduced-motion` — when set, the previous lines do not slide; they jump. - **States:** every interactive element has hover, focus, active, disabled. Loading uses skeletons not spinners. Empty states have specific guidance ("Tap _Start cooking together_ to begin", "Add a cousin to share this recipe"). ## 8. Content generation rules - Write **realistic, specific copy**. NO Lorem Ipsum. NO generic placeholders like 'Your tagline here'. - Invent plausible names, dialects, dishes, and quotes that fit the domain (use the seed content in section 8a as a starting point). When inventing, lean on real twentieth- and twenty-first-century diaspora kitchen patterns — Cantonese steamed fish from Guangzhou, Tamil kuzhambu from Chennai, Salvadoran pupusas, Filipino adobo, Polish pierogi, Lebanese kibbeh — but never claim a fictional cook is a real person. - Tone: warm, direct, free of corporate language. This template is for a family, not a company. - Headlines: punchy and concrete. No 'Empower your X' filler. No 'Revolutionize'. No 'Seamless'. - Body copy: short paragraphs (2-4 sentences). Use lists where appropriate. - Plain language. Avoid jargon — except where the user already speaks the jargon (the heritage-language learner wants to see "_yue-Hant-HK_" in the dialect chip, not "_Chinese (advanced)_"). - Where the app outputs AI-generated content, never label it as "AI says" — let it speak naturally. Use small uncertainty cues only where epistemic honesty requires them (a low-confidence word in the live tile is faintly underlined; tapping it reveals the alternates the model considered). ## 8a. Seed content (use these specific examples) Anchor every generated copy + sample data point in the concrete content below. Use these names, numbers, dates, and snippets verbatim where helpful, or generate close variants that sit in the same world. **Sample families (sidebar):** - "Wong family — Sydney + Guangzhou" (1 cook: Nai-Nai Wong Mei-Ling; 7 recipes; contributors: me, my brother Jonathan in Melbourne, our mum Lily) — Cantonese (Guangzhou register), with English postal-Cantonese hybrids; Saturday-afternoon class format, May 2026 to present. - "Iyer family — London + Chennai" (1 cook: Paati Iyer Lakshmi; 14 recipes; contributors: me, my dad in Edinburgh, my cousin Anjali in Toronto) — Tamil (Chennai register) with heavy English code-switching for kitchen tools; weekend-and-holidays format, March 2026 to present. - "Ramos family — San Francisco" (1 cook: Abuela Carmen Ramos; 9 recipes; contributors: me, my mum, my tía in San Salvador) — Salvadoran Spanish with K'iche' inserts for corn-related ingredients; Sunday-morning format, January 2026 to present. - "Adeyemi family — Lagos + Manchester" (1 cook: Mama Bisi; 6 recipes; contributors: me, my sister, my aunty in Lagos) — Yoruba with frequent English loanwords for measurement, February 2026 to present. **Sample session in detail view (this is what the demo should show):** - **Cook (verbatim):** "Nai-Nai" / "嫲嫲" - **Cook full name:** Wong Mei-Ling, 79, the grandchildren's paternal grandmother - **Source language detected:** Cantonese (yue-Hant-HK), Guangzhou register - **Dialect confidence at detection:** 0.91 - **Session started:** 2026-05-16 15:42 Sydney time - **Session duration:** 47 minutes 12 seconds - **Dishes detected (1):** "薑蔥蒸魚" — "ginger and spring onion steamed fish" — Guangzhou-style steamed fish - **Dish name romanised:** "gēung chūng jīng yú" - **Serves estimate:** "enough for the four of us" - **Ingredients (verbatim):** - "一條新鮮魚, 唔好太大" → "one fresh fish, not too big" (produce; quantity: "一條 (one)"; unit: null) - "兩片生薑, 切到好薄" → "two slices of ginger, sliced very thin" (spice; quantity: "兩片"; unit: "片 (slices)") - "三條蔥, 切段" → "three spring onions, cut into segments" (produce; quantity: "三條"; unit: "條 (stalks)") - "兩湯匙生抽" → "two tablespoons of light soy sauce" (condiment; quantity: "兩湯匙"; unit: "湯匙 (tablespoon)"; modern_conversion_note: null — tablespoon already legible) - "少少糖" → "a little sugar" (sweetener; quantity: "少少 (a little)"; unit: null) - "啲油" → "some oil" (fat; quantity: "啲 (some)"; unit: null) - "Maggi cube 一個" → "one Maggi cube" (condiment; brand-specific: true; quantity: "一個 (one)"; unit: null) - **Steps (excerpt, first 3 of 7):** - **Step 1.** Action source verbatim: "先攞條魚, 洗乾淨佢, 用紙巾抹乾". Action translation: "First take the fish, wash it clean, and pat it dry with kitchen paper." Sensory cue: "until the skin is dry to the touch". Tips attached: "_paati's note: don't skip the drying — wet fish steams unevenly_". - **Step 2.** Action source verbatim: "薑切片, 唔好厚過張紙". Action translation: "Slice the ginger, no thicker than a sheet of paper." Tips attached: "_paati's note: the ginger has to be paper-thin; if it is thick it tastes harsh_". - **Step 3.** Action source verbatim: "蒸八分鐘, 唔好開蓋, 你信我". Action translation: "Steam for eight minutes; don't open the lid, trust me." Time estimate seconds: 480. Doneness cue: "don't open the lid — trust the timer". - **Code-switches (1):** phrase verbatim "Maggi cube", language inserted "en", language carrier "yue-Hant-HK", translation_or_gloss "Maggi cube (a chicken-bouillon brand)", is_loanword_kept: true. - **Tips (3):** - "千祈唔好炒到燶" → "never let it scorch", category "doneness_cue", attached to step 4. - "我媽媽教我嘅" → "this is the way my mum taught me", category "family_lore", attached to step 1. - "你哋姑媽煮就會落多啲糖" → "your aunt would use more sugar", category "family_lore", attached to step 5. - **Flagged for user review (1):** field "ingredients[5].quantity_verbatim", reason "she said _啲_ ('some') without specifying — kept verbatim, no conversion". - **Lineage note:** "this is the way Nai-Nai's own mother made it in Guangzhou before the war; she changed nothing" **Sample input artefacts (for the build to demonstrate):** - A 47-minute recorded session of a Cantonese grandmother teaching ginger-and-spring-onion steamed fish to her two Sydney-born grandchildren, with two long pauses (deboning and washing) and one English code-switch (Maggi cube). - A 92-minute Tamil session of a paati teaching three dishes in one afternoon — rice, fish kuzhambu, thoran — with frequent English code-switches for kitchen tools (pressure cooker, induction, blender). - A 31-minute Salvadoran Spanish session of an abuela teaching pupusas, with two K'iche' code-switches for masa ingredients. - A 15-minute Yoruba session of Mama Bisi teaching jollof, where she uses Maggi cube and "tablespoon" as English loanwords throughout. - A 22-minute Hindi session of a dadi teaching arhar dal with a chhatak-ghee finishing tarka, with the modern conversion ("≈ 60 g") appearing on first chhatak mention only. **Sample voice copy:** - Onboarding: "Your grandmother speaks. You cook. We translate, live." - Pre-session: "Tap when she starts. We'll listen, translate, and write the recipe down for you." - Processing: "Listening to her…" / "Detecting dialect…" / "Translating…" / "Saving the moment…" - Empty archive: "This kitchen is waiting for its first session. Tap _Start cooking together_ when she is ready." - Error (dialect uncertain): "We're not sure if that's Cantonese (Guangzhou) or (Hong Kong). Tap to pick — or keep going and we'll auto-detect after a minute." - Save confirmation: "Saved to Wong family — Nai-Nai's ginger and spring onion fish." - Code-switch detected: "She used a Tamil word here — we kept it for you." - Pause held: (no message — the tile holds her last line, silently. The kitchen is allowed to be silent.) - Heirloom prompt: "Save this session as a family heirloom? The audio and recipe will be preserved forever in your family archive." - Consent prompt (voice clone): "Before we can use [Cook]'s voice anywhere else, we need her to record one sentence saying she agrees. Hand the phone to her." **Sample family invitation email subject + body:** - Subject: "Anjali — I cooked paati's kuzhambu today. Adding it to our family archive." - Body: "Hi Anjali — paati taught me her kuzhambu this afternoon. I've saved the recipe and her voice. Tap to join the family archive — you'll see the recipe, and you can press play on any step to hear her saying it. [Open Archive]" ## 9. Media & assets - **Hero image (landing screen):** A photographed-looking shot of a phone propped on a wooden kitchen counter beside a chopping board with a few slices of ginger and a knife at rest. Warm late-afternoon light coming sideways. The phone screen is on but the content is intentionally blurred to anonymity. Generate via Nano Banana 2 with a prompt emphasising "wooden counter, warm late-afternoon side light, small ginger pieces and a knife at rest, a phone propped on a small wooden stand, no people, no text on the phone, soft shadow under the chopping board". - **App icon / wordmark:** Set in the warm humanist sans. Slightly warm cream background. No icon — just type. - **Empty-state illustration:** A simple line drawing of a phone on a counter beside a small pot, with a single steam curl rising. Hand-drawn aesthetic, not a flat icon. - **Demo session photos:** A 6-photo series captured by the demo grandchild during the Cantonese session: (1) the whole fish on a plate, (2) sliced ginger on a chopping board, (3) the fish in the steamer mid-cook, (4) the steamer lid closed with steam visible, (5) hot oil being poured over scallions, (6) the finished dish on a serving plate. Generated via Nano Banana 2 with the warm-late-afternoon palette of the rest of the app. Each demo photo should look like a phone snap, not a glossy food magazine render. - **Recipe-card hero illustrations:** Per the Nano Banana 2 prompt in 4b. Watercolour-leaning, no text, no people, restrained palette. - **Stock fallbacks:** If image generation fails, fall back to the photographed sample image from `/public/samples/sample-counter.jpg`. Never to a "🍳" emoji. - **Generated imagery:** prefer Nano Banana 2 over stock photography. Prompt for warmth, asymmetry, and slight imperfection — avoid the glossy 'AI render' look. - **Optimisation:** WebP/AVIF, `loading="lazy"`, explicit `width`/`height` to prevent layout shift. - **Icons:** `lucide-react` for UI. Use sparingly — never decorative-only. ### Build-time asset manifest (explicit specs) Every image, illustration, and visual reference mentioned above must resolve to ONE of the three buckets below — runtime-generated, seed-shipped, or user-supplied. Do NOT ship `` tags whose `src` is not listed here. Do NOT depend on bare "section 8a prompts" without binding them to explicit paths and model IDs. **Bucket 1 — Runtime-generated (Nano Banana Pro `gemini-3-pro-image` for hero/demo photographs; Nano Banana 2 `gemini-3.1-flash-image` for in-app illustrations and reference-conditioned variants).** Cached to Firebase Storage; served via signed URL. Every reference above to "Nano Banana 2" or "Nano Banana Pro" MUST be wired to one of these specific calls with an explicit model id: - `/public/generated/hero.webp` (2400×1500, WebP) — model `gemini-3-pro-image` — uses the literal prompt described as "Hero image (landing screen)" above. Run once at build; commit a `/public/samples/hero-fallback.webp` (1600×1000) generated from the same prompt with `gemini-3.1-flash-image` so the page renders if quota is exhausted. - `/public/generated/demo/{demo-slug}-{NN}.webp` (1600×1200, WebP) — model `gemini-3.1-flash-image` (reference-conditioned where the prior frame is passed as input) — one path per "Demo X" image referenced above. The slug derives from the seed example in section 8a; the NN index covers each frame in the demo sequence. - `/public/generated/illustrations/{name}.webp` (1024×1024, WebP) — model `gemini-3.1-flash-image` — one path per named illustration above ("Empty-state illustration", "Recipe-card hero illustrations", "Curriculum picker imagery", "Period-style frames", etc.). Each illustration's prompt is the literal description above; ship a deterministic seed in the request so re-runs are reproducible. **Bucket 2 — Seed assets shipped with the deliverable.** Every "Stock fallback" path referenced above (e.g. `/public/samples/sample-X.jpg`) is generated once via Nano Banana 2 (`gemini-3.1-flash-image`) at 1024×1024 WebP using the same prompt as its Bucket-1 counterpart, then committed to the repo so the page renders identically if Gemini quota is exhausted or the user is offline. Replace any `.jpg` extension above with `.webp` to match the optimisation rule. Also commit these empty-state seeds (1024×1024 WebP, single-stroke hand-drawn line, no colour fill): - `/public/samples/empty-state-primary.webp` — line drawing of the app's primary empty surface (the named "Empty-state illustration" above), generated from that exact prompt. - `/public/samples/empty-state-archive.webp` — line drawing of an empty saved/archive view, single-stroke outline. - `/public/samples/empty-state-error.webp` — line drawing of a hand placing a single object aside with care, used when an AI call fails. **Bucket 3 — User-supplied.** Uploads from the user's camera / file picker land at the Firebase Storage path conventional for this template (named in section 4b). The build ships with Bucket-1 + Bucket-2 only; no user-supplied images at first paint. **Hard rules** - Every `` tag MUST have a `src` that resolves to a path listed in Bucket 1, Bucket 2, or a Bucket 3 upload path. Anything else is a build error. - No bare `image.jpg` / `hero.jpg` / `placeholder.png` references anywhere in the code. - Model IDs: `gemini-3-pro-image` for hero-quality photographic generation; `gemini-3.1-flash-image` for in-app illustrations, reference-conditioned variants, empty-state seeds, and stock fallbacks. Never use a legacy model id (no `imagen-*`, no `gemini-1.5-*-image`). - File format: WebP everywhere (AVIF acceptable where the target browsers support it). No `.jpg` / `.jpeg` / `.png` in `/public/samples/`. ## 10. Interactivity & states - Every interactive element has hover, focus, active, and disabled states. - Forms validate inline and show specific error messages (not "Invalid input"). - Loading states use skeletons that match the eventual layout, not spinners. - Empty states explain the next action with a button whose label fits THIS app's domain: "Start cooking together", "Add a cousin to share this with", "Save as family heirloom" — never a generic "Add your first item". - Smooth scroll for in-page anchors. - Live API text streams in token-by-token into the counter tile. A subtle "listening…" indicator (a small breathing dot, not a spinner) appears between her clauses; never on top of her words. - Live API audio playback (when on) starts only after she has finished a clause; if she resumes speaking, the audio cuts cleanly (with a brief 40 ms fade) and the next translation streams in. - If the Live API call fails or the connection drops, show a calm, specific error ("We lost the connection. We're recording her voice locally — when you're back online, the recipe will pick up exactly where you left off") and queue the audio for the offline-fallback transcript. - Low-confidence words in the live tile are faintly underlined; tapping reveals the alternates the model considered. - The recipe-card drawer slides out from the side of Counter Mode at 240 ms; with `prefers-reduced-motion`, it appears instantly. - The screen wake-lock is acquired when the Live session starts and released when it ends. On platforms that do not expose wake-lock, fall back to keeping the audio context active (which prevents most mobile-browser sleep). - Tapping "hand back" mid-session toggles the translation off (the tile dims and a small "translation paused — still recording" chip appears at the bottom); tapping it again resumes. **Session-resume snippet (Live API 2-min cycle):** the Live API audio+video session caps at 2 minutes. On every Live tick, persist a `SessionSyncState` to `sessionStorage`; on reconnect, pass a concise context-summary block as the first system message of the next handshake so the model continues without losing thread. ```typescript interface SessionSyncState { activeSessionId: string; accumulatedSegments: Array<{ speaker: string; text: string; timestamp: number }>; // ...template-specific cursor state (current page, turn index, etc.) } ``` ## 11. Tech & responsive requirements - **TTS markdown-stripping preprocessor:** before sending any user-authored markdown to `gemini-3.1-flash-tts-preview`, strip non-spoken markdown: `#`/`##`/`###` headings (keep the title text), `**bold**` (keep the inner text), `[label](url)` (keep `label`, drop URL), `` ``` `` fenced code blocks (skip entirely), `>` block-quote markers (keep the text), and `|` table pipes (read row-by-row as sentences). Insert `…` between sentences for a short pause and a blank line plus `—` between paragraphs for a long pause. The model does not understand markdown; raw markdown will be read aloud as literal characters ("asterisk asterisk"). - **File downloads on Safari / Firefox:** when offering local-disk save of any export (PDF, CSV, MP3, ZIP, JSON, image), fall back to `` with a blob URL — the File System Access API (`showSaveFilePicker()`) is Chromium-only. Detect with `'showSaveFilePicker' in window`; otherwise use the anchor-download path. - **Stack:** React + TypeScript + Tailwind CSS. Functional components + hooks. Use Shadcn UI primitives where appropriate. - **Build runtime:** AI Studio Build — full-stack with Cloud Run server-side functions. All Gemini API calls happen server-side; API key lives in Secrets Manager, never in client bundle. The Live API session is brokered through a server-side WebSocket relay so the API key never reaches the client. - **Model selection:** explicitly pin `gemini-3.1-flash-live-preview` for the Live session; `gemini-3.5-flash` for session-end structured extraction and photo confirmation; `gemini-3.5-flash` for per-step parse and dialect detection; `gemini-3.1-flash-tts-preview` for TTS; `gemini-3.1-flash-image` for the recipe-card hero illustration. Set `thinkingLevel` explicitly per call where supported. - **Database:** Firestore (auto-provisioned by AI Studio Build). Show the seed sample family on first launch. - **Auth:** Firebase Auth — Google sign-in by default; Apple sign-in next to it; magic-link email as fallback. - **Storage:** Firebase Storage for session audio and step photos. Pre-signed URLs only. - **Mobile-first.** Verify layouts at 375 px (iPhone SE), 768 px (iPad), 1024 px, 1440 px+. Counter Mode is the primary mobile experience; the recipe-card detail view is the primary tablet/desktop experience. - Use `clamp()` for fluid typography on the Counter Mode tile (clamp 28 px → 64 px) so the translated line reads across the counter on phone or tablet. - Prefer container queries over media queries for component-level responsiveness — the Counter Mode tile, the recipe-card detail view, and the family-archive list each adapt to their container, not the viewport. - Use `dvh` / `svh` instead of `vh`. Respect safe-area insets on iOS. - Zero horizontal overflow at any width. Zero layout shift on load. - Persist user data in Firestore. Use real-time listeners on the family-archive view. - Optimistic UI on writes; reconcile on response. - The Live API session opens a WebSocket from the client to a Cloud Run function which proxies to the Gemini Live endpoint; the client streams microphone PCM in and receives translated text + optional audio out. - **iOS Safari gotchas (graceful degradation):** the Live API session must survive iOS audio-session interruption (incoming call, Siri, alarm) — listen for `MediaStreamTrack.onmute` and pause; resume on `onunmute`. Mic permission does NOT persist across page reloads on iOS — re-request at the start of every cooking session. Backgrounded Safari tabs throttle WebSocket and kill `getUserMedia` — pair `visibilitychange` with a screen Wake Lock during a session so the grandmother's voice isn't dropped when the grandchild peeks at a message. PCM streaming must go via `AudioWorklet` (Safari `MediaRecorder` is AAC-only). - Browser microphone permission is requested only at "Start cooking together" — never at app load. ## 12. Accessibility (WCAG 2.2 AA) - Semantic HTML — `header`, `nav`, `main`, `section`, `article`, `footer`. - All interactive controls reachable by keyboard with a visible focus ring. - Color contrast ≥ 4.5:1 for body, 3:1 for large text and UI components. The Counter Mode translated line meets a stricter 7:1 contrast against the warm cream background, because it has to be readable across a kitchen counter. - All images have meaningful `alt` text. The session step-photos have `alt` describing the moment ("photograph of the fish on a chopping board, ginger slices to one side, captured during step 2"). - Form fields have associated ``s; errors announced via `aria-describedby`. - All modals: focus trap, `Esc` to close, `role="dialog"`, `aria-modal="true"`, `aria-labelledby`, restore focus to trigger on close. - Respect `prefers-reduced-motion`. The counter-tile line-slide animation is the canonical reduced-motion check — verify it jumps instead of sliding. - Tap targets ≥ 44 × 44 px on touch. The Counter Mode floating controls (audio toggle, capture photo, hand back, end session) are 56 × 56 px to be hittable with a wet finger. - Live API output is announced to screen readers via a polite `aria-live="polite"` region for the translated tile; the previous-line history is `aria-live="off"` to avoid double-reading. - TTS playback exposes a transcript that scrolls in sync; a 'pause to read this passage' button stops at any paragraph. - Low-confidence underlines are also announced as "uncertain reading" to screen readers via `aria-describedby`. - Recipe-card detail view exposes a list-mode alternative ("read recipe as a flat list") for users who navigate primarily by keyboard or screen reader. - The session-end review screen is fully keyboard-navigable, including the inline edit-any-field flow. ## 13. Quality bar — avoid AI clichés Do **not** ship any of the following unless the design explicitly calls for it: - Hero with a gradient background and floating geometric / orb shapes. - A row of 4 'by the numbers' stat cards below the hero ("47 minutes, 7 ingredients, 1 grandmother, 1 dish"). - Emoji bullets in body copy. - 'Lorem ipsum' or 'Your headline here' placeholders. - A pricing table with three identical tiers when the product has no real pricing. - Faux testimonials with stock-photo headshots. - Hero CTA labelled 'Get started' with no second action. - "AI" or "Powered by AI" badges plastered everywhere. The intelligence is in the experience, not the marketing. - Loading spinners on AI calls — show a meaningful 'listening' state (a small breathing dot, never a busy spinner). - Toast notifications for routine actions (save, delete) — use inline confirmation instead. - Any decorative use of ethnic-stereotype imagery in app chrome (claypots-and-spices flat-icon clichés, "authentic" wood textures branded as ethnic-X, period costume art). The cook's own dish and the photographs the grandchild took are the only ethnic-specific content the app shows. - Any "AI chef" persona, "translate your culture" badge, or "preserve heritage" branding. The app does one thing and is honest about it: it translates a grandmother teaching her grandchildren to cook, and it writes the recipe down. Aim for the polish of a hand-crafted production app — specific, considered, with real domain knowledge in the copy. A reviewer should not be able to tell this came from a template. ## 14. Deliverables A single working app I can preview immediately, with: - All sections populated with realistic seed data (see section 8a) — the demo families include the Wong family (Sydney + Guangzhou, Cantonese), the Iyer family (London + Chennai, Tamil), the Ramos family (San Francisco, Salvadoran Spanish), and the Adeyemi family (Lagos + Manchester, Yoruba) - All Gemini features wired to real API calls (server-side) with the per-call matrix respected exactly — Live API for the session, Gemini 3.5 Flash for session-end extraction, Gemini 3.5 Flash for per-step parse and dialect detection, Gemini 2.5 Flash TTS for narration, Nano Banana 2 for the recipe-card hero - Live API session brokered through a server-side WebSocket relay; API key never reaches the client - All interactive states implemented (listening dot, low-confidence underlines, pause held, hand-back chip, offline-fallback recording, error retry) - Counter Mode primary mobile experience with screen wake-lock and clamp() fluid typography reading across a kitchen counter - Onboarding modal working on first visit (and accessible via `?` icon thereafter) - Capabilities info button working in header (`(i)` icon → panel) — fully populated with model names, costs, privacy notes - Auth flow working with Google + Apple sign-in - Firestore persistence wired up with security rules that scope sessions, recipes, and audio to the family owner + invited members - Printable recipe-card PDF export working end-to-end on the demo session - Family-archive share working end-to-end (magic link → cousin in another city sees the recipe and can play the audio) - Voice-clone heirloom flow stubbed but clearly gated behind family-owner opt-in and grandmother-recorded spoken consent - Responsive across the four viewport breakpoints - Zero TODOs, zero console errors, zero placeholders