================ ONE-SHOT BUILD CONTRACT (read first) ================
Build this in Google AI Studio "Build" in ONE shot — a complete, working app,
no follow-up turns. These are hard rules, not suggestions:
1. TARGET = Full-Stack Web (Node server runtime, secrets, Firebase allowed).
If you target Native Android instead, you MUST drop all server/DB/Workspace/
secrets and persist locally (Room / SharedPreferences) only.
2. PIN EVERY MODEL CALL — never let the agent auto-select (it downgrades on quota):
- Reasoning / text -> gemini-3.5-flash (thinkingLevel: minimal|low|medium|high)
- 4K image + legible text -> gemini-3-pro-image (image_size "4K", up to 14 refs)
- High-volume image -> gemini-3.1-flash-image
- Expressive TTS -> gemini-3.1-flash-tts-preview (inline tags e.g. [whispers])
- Realtime audio/video (WebSocket) -> gemini-3.1-flash-live-preview
- Sandboxed agent -> antigravity-preview-05-2026
3. DIVISION OF LABOR — the model ONLY parses/extracts to a strict responseSchema.
ALL math, money (store currency as integer minor units / cents), sorting,
balancing and graph logic run in deterministic TypeScript/Python. The model
must never compute totals, splits or balances itself.
4. responseSchema sanitation — no regex patterns, no fixed-length tuples, no
format validators in the schema (they crash the OpenAPI engine). Enforce those
in server-side code AFTER parsing the JSON.
5. responseSchema and google_search grounding are MUTUALLY EXCLUSIVE in one call.
6. CODEGEN — split large output into modular, single-responsibility files so no
file is truncated by the output-token cap.
7. Every external call gets a graceful fallback (e.g. manual paste if a Workspace
read fails). Never a silent dead end.
8. ROBUST STORAGE & CANVAS — Wrap all `localStorage`/`sessionStorage` operations (especially JSON parsing and writes) in `try-catch` blocks to prevent crashes in private windows or quota overflows. Canvas drawing elements must dynamically handle window resize and scale pixel density (`window.devicePixelRatio`) to avoid blurry graphics on retina displays.
=====================================================================
# MUST OBEY — Mobile-first build requirements
This app's PRIMARY surface is a mobile phone. Build it impeccably on mobile FIRST, then verify on tablet and desktop. Treat the rules below as non-negotiable hard constraints, not suggestions.
## Viewports to verify (every screen, every state)
- 320 px, 360 px, 375 px, 390 px, 414 px, 480 px
- 768 px, 834 px (iPad portrait / Pro 11)
- 1024 px, 1280 px, 1440 px, 1920 px, 2560 px
- Plus: 200% browser zoom, landscape orientation on every mobile width, iPhone with safe-area insets visible
## Hard layout rules
- Mobile-first CSS. Default styles target mobile; `@media (min-width: ...)` for larger viewports.
- Use `dvh` and `svh` instead of `vh` for full-height surfaces (iOS Safari URL-bar bug).
- Use `clamp()` for fluid typography across all viewports.
- Prefer container queries (`@container`) over media queries for component-level responsiveness.
- Use `min(100%, ...)` widths so content never overflows. Zero horizontal overflow at any viewport.
- Add `` to every page.
- Apply `padding: max(safe-area-inset-X, fallback)` on every edge-bleeding container so notched iPhones in landscape never clip content.
- Wide tables and code blocks scroll INSIDE their container (`overflow-x: auto`), never push the body.
- Use `background-attachment: scroll` on mobile, not `fixed` (iOS Safari repaint bug).
- Avoid `backdrop-filter` on animated elements. Use it sparingly on static surfaces only.
- **Canvas Scaling**: Canvases must dynamically scale with window resize events and properly handle high-DPI screens (`window.devicePixelRatio`). Set physical dimensions (`canvas.width`/`canvas.height`) using pixel ratio and render relative to this grid, using CSS to control responsive viewport scaling.
- **Robust Storage**: Every access to `localStorage`/`sessionStorage` (especially `JSON.parse` of loaded state or writes) MUST be wrapped in a `try-catch` block to handle disabled storage, private browsing mode, quota limits, or corrupted JSON gracefully. Fall back to a robust in-memory object store.
## Touch & accessibility
- Tap targets ≥ 44 × 44 px on touch (Apple HIG). Increase to 48 px under `@media (hover: none) and (pointer: coarse)`.
- All interactive controls reachable by keyboard with a visible focus ring; respect `:focus-visible`.
- Color contrast ≥ 4.5:1 for body text, 3:1 for UI components.
- All images have meaningful `alt`. Decorative images use `alt=""`.
- Respect `prefers-reduced-motion: reduce` — zero animation durations under that query.
- Forms validate inline; error messages are specific, not "Invalid input".
- Modals: focus trap, `Esc` closes, `role="dialog"`, `aria-modal="true"`, focus restored on close.
## Performance bar (Lighthouse mobile, throttled 3G/4G)
- LCP < 2.5 s · INP < 200 ms · CLS < 0.1
- JS bundle gzip < 200 KB mobile-first; lazy-load non-critical screens via `React.lazy` / dynamic imports.
- No render-blocking resources above the fold.
- Images: WebP/AVIF preferred, `loading="lazy"`, explicit `width`/`height` attributes (zero CLS), `srcset` for retina.
- Videos: `preload="metadata"`, low-resolution poster, max 720p mobile fallback. Never autoplay with audio.
- Fonts: `font-display: swap`; preload only the one used above the fold.
- Smooth scroll honoured via CSS `scroll-behavior: smooth` with reduced-motion fallback.
## Pre-ship mobile checklist (the deployer MUST verify before declaring done)
1. Open at 375 px in DevTools — every screen scrolls vertically only; zero horizontal scroll.
2. Browser zoom 200% — layout reflows without overlap.
3. iPhone Safari with the URL bar visible AND landscape — no content under the home indicator; no notch clipping.
4. iPad portrait (768 px) and landscape (1024 px) — no awkward gaps; tablet-specific breakpoints land cleanly.
5. Tap every interactive element with a thumb at real-device size — every target is easy to hit.
6. `prefers-reduced-motion: reduce` — every transition / animation skips cleanly, scroll-behavior becomes instant.
7. Lighthouse mobile score ≥ 90 across all 4 categories.
8. Zero `console.error` and zero CLS shift in real-device testing on a mid-tier Android (e.g. Pixel 6a) and an iPhone SE.
---
The original template starts below. All rules above apply on TOP of whatever this template specifies.
---
# Museum Whisper
## 1. Project
**Museum Whisper** is a one-painting-at-a-time audio guide for visitors
standing inside a museum. The user lifts their phone toward whatever
is in front of them — a painting, a bronze, a manuscript page in a
vitrine, a Coptic icon, a Ming dynasty scroll — and ninety seconds
later, headphones in, they have a guided look written specifically
for the work they are facing. Not the room. Not the period. This
object. Where to put their eyes first, where to put them next, the
single piece of context the wall plaque chose to leave out, and the
small detail that, once seen, cannot be unseen.
This is the kind of app a visitor opens at the National Museum of
Ethiopia in Addis Ababa, in front of a thirteenth-century processional
cross, when the wall card is in Amharic and English but stops at the
attribution and leaves the iconography unexplained. It is also the
kind of app a visitor opens at the National Palace Museum in Taipei,
in front of a Northern Song landscape so dense that the eye does not
know where to begin, when the audio guide queue is forty minutes long
and the show closes in an hour. Same shape of moment, different
script on the wall, different century on the object.
The single demo that proves the magic: the visitor opens the app,
points the camera at a painting, and within ninety seconds — without
typing anything, without selecting from a list — hears a guided look
that names the painting, then says "your eyes are probably on the
figure in red — stay there for a moment, then drop down and to the
left to the dog. The dog is what changes the meaning. Here is what
the plaque does not tell you." A small transcript on screen
underlines the three places the voice asks them to look, with a
faint coloured ring drawn over the matching region of the painting
when they tap.
And in the harder cases — works with contested attribution, works
removed from their original ritual context, works with sensitive
provenance, works the museum itself has reframed in the last decade —
the app speaks carefully. It quotes the museum's own current label
where one exists, names the scholarly disagreement where one is
public, and tells the visitor what is settled and what is still in
argument among curators. It never invents provenance, never invents
a sitter's biography, never narrates a "secret history" that is not
in the literature.
**Tagline:** _Stand in front of one painting and hear what to look at next — in any museum, any script, any room._
## 2. Target audience
- Adult museum visitors with thirty to ninety minutes total and one room they want to do well, rather than a whole wing badly
- First-time visitors to a country's national museum on the first morning of a trip — Addis Ababa, Taipei, Mexico City, Tashkent, Dakar, Hanoi, Lima, Tehran (when accessible), Cairo, London, Madrid
- Parents trying to hold a nine-year-old's attention in front of one object for ninety seconds instead of moving through three rooms in five minutes
- Art-history students reviewing for an exam, who already know the period and want the contested reading, not the wall-plaque reading
- Practising artists studying a single technique — Caravaggio's hand lighting, Sesshū's broken ink, Amrita Sher-Gil's flesh tones — who want to be told exactly which brushstroke to look at
- Visitors whose first reading language is not the language on the wall labels — a Tamil-speaking visitor at the Louvre, a Vietnamese-speaking visitor at the Rijksmuseum, a Swahili-speaking visitor at the Met, a Farsi-speaking visitor at the British Museum
- Low-vision visitors who want a longer, more spatially precise description than the museum's printed audio guide offers
- Visitors with sensory or social-energy limits who prefer a quiet phone-and-headphone experience to a guided tour with strangers
- Museum members and docent-trainees who want to compare what the app says to what they were trained to say, and to flag disagreements back to curators
## 3. Core value propositions
Surface these clearly through copy, visual emphasis, and section ordering — they are the reasons users pick this app.
- **One object at a time, ninety seconds at a time** — the app refuses to give a tour. It gives a single, slow look at the thing in front of you. The visitor leaves the app between objects, walks, looks, lifts the phone again at the next thing that catches them. No tour route, no completion percentage, no gamification.
- **Identifies the object from a phone-at-the-painting photograph** — Gemini 3.5 Flash reads what is on screen: the work itself, the wall plaque if visible, the room context. It returns a best-guess attribution and a confidence level, asks the visitor to confirm, and refuses to bluff. If two paintings in the museum's collection look similar from a phone snap, the app surfaces both and lets the visitor pick.
- **Grounded art-historical context, never invented** — Gemini 3.5 Flash with `google_search` grounding fetches the museum's current label, the catalogue raisonné entry where one is on the open web, and the most-cited recent scholarship on the work. Every claim in the spoken script ties to a source URL the visitor can open. If a claim has no source, the script does not make it.
- **A guided look, not a description** — the script is structured around eye movement: where to look first, where to look next, where the meaning sits, what to notice in the corner you would have missed. The voice points. It does not lecture.
- **One piece of context the plaque omitted** — every script includes one carefully-chosen fact the museum's own label does not say. Sometimes it is technical (the underdrawing visible in raking light), sometimes provenance (where the work was between 1933 and 1947), sometimes interpretive (what the figure in the corner is doing, in the iconography of that century). The app earns its existence on this fact.
- **Respects the work's current framing** — if the museum has updated the label in the last decade to address attribution, provenance, or colonial context, the app uses the current framing, not the older one. Where a label has been replaced, the app says so and quotes both.
- **Speaks the visitor's language** — the spoken guide is generated in the visitor's chosen language, with Gemini 2.5 Flash TTS narration in a voice native to that language. The visitor at the Louvre listening in Tamil hears Tamil; the visitor at the Met listening in Swahili hears Swahili.
- **Save the look, not the score** — the visitor can save the ninety-second guided look to a small personal archive: the photograph they took, the script, the audio, the sources. No leaderboard, no streak, no badge.
## 4. Features to build
- Camera capture pointed at the object (single tap), with a gentle frame guide that suggests "fill the frame with the work and a corner of the wall plaque if visible"
- Two-shot mode for crowded rooms: a wide shot of the object + a close shot of the wall plaque, both fed into the same identification call
- Object identification from the photograph(s) with confidence level, alternates if ambiguous, and a "tell us which one" confirmation step
- "I already know what this is" path: the visitor types the artist + title and skips identification entirely
- Live grounded fetch of: the museum's current label/online catalogue entry, the work's catalogue raisonné (where on the open web), the most-cited recent paper on the work
- Structured 90-second guided-look generation: opening orientation (10s), eyes-first cue (15s), eyes-next cue (20s), one omitted-context fact (25s), one corner-detail (15s), closing (5s)
- TTS narration in the visitor's chosen language with Gemini 2.5 Flash TTS, at a museum-floor pace (slightly faster than letter-reading pace because the visitor is standing)
- On-screen transcript that scrolls in sync with audio; each "look here" cue is tappable and highlights a region on the captured photograph
- Region overlay rendering — when the script says "the dog in the lower left", a soft circular ring appears over the corresponding region on the captured image
- Source panel — every spoken claim is footnoted with a tap-to-open URL drawn from `groundingMetadata.groundingChunks[].web.uri`
- "Disagreement" surface — when scholars publicly disagree about a work (sitter identity, attribution percentage, date), the app surfaces both readings rather than picking
- "What the plaque does not say" toggle — show the omitted-context fact more prominently, with its source
- Personal archive: save the photograph + script + audio + sources for any guided look. Reorder, rename, share by link
- "Pair with a friend" — a second visitor in the same room can pair phones via short code and hear the same guide simultaneously, in different languages
- Quiet mode — works on poor museum Wi-Fi by batching only the identification call; the rest streams when signal returns
- Offline catalogue — a small bundled list of a few hundred frequently-visited works at major museums, for which a pre-generated script is available without network
- Accessibility long-description mode — extends the 90s to ~180s, with denser spatial language, for visitors who cannot easily see the work
- Wall-plaque OCR fallback — if identification by image alone is uncertain, the visitor photographs the plaque and the app reads the museum's own attribution
- Language picker that remembers per-visit and respects device locale on first launch
- Visit history view — every guided look from the day, ordered as the visitor walked, with thumbnails
## 4b. Required Gemini capabilities + backend services
**This template's intelligence comes from the Gemini capabilities below. Wire them up explicitly — don't substitute generic LLM calls.**
### Gemini capabilities (the load-bearing intelligence)
- **Multimodal image input** (Gemini 3.5 Flash) — identifies the work from a phone photograph taken under museum lighting (mixed warm tungsten, indirect daylight, sometimes glass-on-painting glare). Reads the wall plaque when partially visible. Recognises the framing, the room context, and approximate scale from the photograph.
- **Grounded search** (`google_search` enabled on Gemini 3.5 Flash) — pulls the museum's current online label, the catalogue raisonné entry where on the open web, and recent scholarship URLs. Grounding citations come from `response.groundingMetadata.groundingChunks[].web.uri`. **Hard constraint**: `responseSchema` and `google_search` cannot be combined in a single Gemini call. The identification + grounded-research call therefore emits JSON in the text body and the server parses it; structured-output calls (the 90-second script schema, the personal-archive save) run on a separate, ungrounded follow-up call that takes the grounded text as input.
- **Structured output / JSON Schema** — the 90-second guided-look response matches the `GuidedLook` schema below. Every region cue is typed; the schema is included as `responseSchema` on the script-generation call (which is NOT grounded, so the schema is allowed).
- **Multilingual generation** (built into Gemini 3.5 Flash) — composes the script in the visitor's chosen language. Supports Tagalog, Vietnamese, Mandarin (Traditional and Simplified), Cantonese, Korean, Tamil, Hindi, Urdu, Bengali, Punjabi, Amharic, Swahili, Farsi, Khmer, Arabic, Hebrew, Greek, Russian, Polish, German, French, Italian, Spanish, Portuguese, English, Quechua (where the visitor selects it).
- **Long context (1M tokens)** — only used when a single work has an unusually long literature (Mona Lisa, Las Meninas, the Sistine ceiling). The grounded-research call accumulates ~30-60 source extracts; the script generation call sees them all. For typical works the context is comfortably under 50k tokens. **Guardrail**: if the accumulated context exceeds 800k tokens (rare; happens for the most-written-about works) chunk by source type — primary museum sources first, then catalogue raisonné, then recent scholarship — and produce the script in two passes.
- **Gemini TTS** (`gemini-3.1-flash-tts-preview`) — narrates the 90-second script in the visitor's chosen language. Voice is picked by `languageCode`; pronunciation follows the voice's native locale. The script is generated with TTS-friendly pacing cues already inserted: `…` for short pauses, blank lines plus `—` for longer pauses.
- **Thinking levels** — `medium` for the identification + grounded-research call (the model has to weigh photograph + plaque OCR + multiple search results before naming the work). `low` for the script generation call (the work is now known; this call composes prose to the schema). `low` for the per-language re-narration call. Surface `thoughtSummary` only when the visitor taps "how did the app decide what this is".
### Backend services
- **Auth — Optional but recommended.** Firebase Auth with Google sign-in (auto-provisioned by AI Studio Build). The app works fully without auth (anonymous Firebase Auth backs the personal archive on-device). Signed-in users get cross-device archive sync. **Apple sign-in** is optional and user-configured: it requires an Apple Developer account, Service ID, Key ID, and private key wired into the Firebase Auth console. **Magic-link email** sign-in similarly requires the sender domain to be authorised in Firebase Auth.
- **Database — Required.** Firestore for `users`, `visits`, `guided_looks`, `sources`. Guided-look records are private to the visitor by default.
- **File storage — Required.** Firebase Storage for the visitor's captured photographs and the generated TTS audio files. **Storage is NOT auto-provisioned by AI Studio Build today** — enable it in the Firebase console and wire the bucket name into the AIS Build project before the first capture. Pre-signed URLs only; photographs are never publicly addressable. Audio is cached per (work, language) pair to avoid regenerating common combinations.
- **Email — Not required for v1.** If the visitor opts to share a guided look by link, the link is generated client-side and shared via the device share sheet. No transactional email.
- **Payments — Not needed for v1.** Free for personal use. A possible later tier: a museum membership upgrade that unlocks high-quality, multi-language pre-generated scripts for the museum's full collection — paid for by the institution, not the visitor.
- **External APIs:** Gemini API for all intelligence. No museum APIs are required to ship v1; the museum's own website is what `google_search` grounding reaches. Optional v2 integrations: the Smithsonian Open Access API, the Rijksmuseum API, the Cleveland Museum of Art API — these are public and would let the app render high-resolution reference imagery alongside the visitor's photograph, but are not load-bearing.
**Environment variables:** every secret (Gemini API key, Firebase service-account JSON) lives in environment variables — never in client bundle. Include a `.env.example`.
**Auth + data privacy reminders:** never log secrets · never store passwords in plain text · use HTTPS everywhere · honour 'delete my visit' inside the UI · explicit opt-in for any analytics · the visitor's captured photographs are private to them · this app uses the Gemini API on the paid tier, where Google does not use your content for model training, per the Gemini API Additional Terms · the app never reveals the visitor's location or visit history to anyone, including the museum, without explicit consent.
**Read this first — prompt-craft rules that apply to every call in this template:**
1. **Name the model variant explicitly** in every Gemini API call. Do not let the agent pick the model. See the per-call matrix below.
2. **Pin `thinkingLevel` explicitly** per call. See the matrix.
3. **Seed the JSON Schema as a fenced TypeScript / Zod block** in the system instruction or `responseSchema` field. The literal schema is below. **Convert the Zod schema to Gemini's `Schema` type via the SDK helper** before passing to `responseSchema` — do NOT pass raw Zod. **Numeric `min`/`max` constraints are documentation only inside `responseSchema`; clamp on the server after the response arrives.**
4. **Pin the system instruction separately** from user input. Use the `systemInstruction` field for persona + behavioural rules; use `contents` for user input. Never concatenate.
5. **Pre-declare tools as an enable/disable list** per call. The matrix below names which tools are enabled per call. Tools NOT listed for a call should be disabled.
6. **State negative constraints explicitly** — they are listed below. They are NOT "be careful" suggestions; they are hard rules the model must follow.
7. **Grounding and structured output are mutually exclusive in a single call.** When a call uses `google_search` grounding, do NOT set `responseSchema`. Instruct the model to emit JSON in the text body and parse it server-side. Read citations from `groundingMetadata.groundingChunks[].web.uri`, never from the JSON body the model produced.
8. **Grounded responses can wrap JSON in ```json fences or add prose preamble.** Server-side, strip fences and brace-extract:
```typescript
function safeExtractJSON(raw: string): T {
const clean = raw.replace(/```json\s*|```/gi, '').trim();
const s = clean.indexOf('{'); const e = clean.lastIndexOf('}');
if (s === -1 || e === -1) throw new Error('No JSON boundaries in grounded response');
return JSON.parse(clean.slice(s, e + 1)) as T;
}
```
9. **Strip unsupported Zod modifiers before passing to `responseSchema`** — Gemini's OpenAPI subset rejects `.regex()` / `pattern`, fixed-length `z.tuple()`, and other custom validators. Use a sanitizer that flattens tuples to length-2 arrays and removes regex patterns before serializing. Validate those constraints in middleware AFTER parsing.
### Per-call model + tools matrix
| Call | Model | thinkingLevel | Tools enabled |
|------|-------|---------------|---------------|
| Identify the work from photograph(s) + plaque OCR + grounded research | `gemini-3.5-flash` | medium | `google_search` (no `responseSchema` on this call — see note) |
| Generate the 90-second guided-look script → `GuidedLook` schema | `gemini-3.5-flash` | low | (none) — takes grounded text from prior call as input |
| Re-narrate a script into a different visitor language | `gemini-3.5-flash` | low | (none) |
| Generate TTS audio of the script | `gemini-3.1-flash-tts-preview` | n/a | n/a |
| Long-description (accessibility) mode — extend script to ~180s | `gemini-3.5-flash` | low | (none) |
| Resolve disagreement between two label versions (when both surface) | `gemini-3.5-flash` | low | (none) |
*Note for builders:* on TTS calls, omit `thinkingConfig` entirely — the field is not supported on `gemini-3.1-flash-tts-preview`. The `n/a` cells in this matrix are documentation only; do not serialise them into the request body.
### Primary structured-output schema (seed this verbatim in the prompt)
```typescript
import { z } from "zod";
const RegionCue = z.object({
bbox_normalised: z.object({
x: z.number().min(0).max(1),
y: z.number().min(0).max(1),
w: z.number().min(0).max(1),
h: z.number().min(0).max(1),
}),
label_short: z.string(), // "the figure in red", "the dog"
spoken_phrase_verbatim: z.string(), // the phrase from the script that points here
shape_hint: z.enum(["circle", "rect", "polygon"]),
});
const ScriptBeat = z.object({
beat_id: z.enum([
"opening_orientation", // 10s — what is this object, in one sentence
"eyes_first", // 15s — where to look first
"eyes_next", // 20s — where to look next, why
"omitted_context_fact", // 25s — the one thing the plaque does not say
"corner_detail", // 15s — the detail you would have missed
"closing", // 5s — a thing to carry with you to the next room
]),
text: z.string(), // the verbatim spoken text
approx_duration_seconds: z.number().min(3).max(40),
region_cues: z.array(RegionCue), // 0+ regions referenced by this beat
sources_used_ids: z.array(z.string()), // ids into Source[] below
});
const Source = z.object({
source_id: z.string(), // local id, e.g. "S1"
url: z.string(), // from groundingMetadata
publisher_or_institution: z.string(), // "The National Gallery, London", "Brill", "Kunsthistorisches Museum"
title: z.string(), // page title as fetched
used_for: z.array(z.string()), // ["attribution", "provenance", "iconography"]
is_museum_primary_source: z.boolean(),
});
const Attribution = z.object({
artist_verbatim_as_labelled: z.string(), // exactly as the museum currently labels it
artist_canonical: z.string().nullable(), // common form: "Diego Velázquez"
title_as_labelled: z.string(),
title_alternate_known: z.array(z.string()), // "Las Meninas", "The Family of Philip IV"
date_as_labelled: z.string(), // verbatim: "c. 1656"
date_iso_earliest: z.string().nullable(),
date_iso_latest: z.string().nullable(),
medium_as_labelled: z.string(),
dimensions_as_labelled: z.string(),
current_location: z.object({
institution: z.string(), // "Museo del Prado"
room_or_gallery: z.string().nullable(),
city: z.string(),
country: z.string(),
}),
attribution_confidence_scholarly: z.enum([
"uncontested",
"majority_attributed",
"workshop_or_circle",
"contested",
"after_or_imitator",
]),
});
const DisagreementNote = z.object({
topic: z.string(), // "identity of sitter", "date range", "workshop participation"
position_a_summary: z.string(),
position_a_source_id: z.string(),
position_b_summary: z.string(),
position_b_source_id: z.string(),
museum_current_position_summary: z.string().nullable(),
});
const OmittedContextFact = z.object({
category: z.enum([
"technique_visible_in_work",
"iconography_decoded",
"provenance_period",
"patronage_or_commission",
"removal_from_original_context",
"conservation_history",
"label_changed_in_last_decade",
"scholarly_disagreement",
]),
one_sentence_fact: z.string(), // the spoken sentence verbatim
source_id: z.string(), // mandatory; no source = no fact
why_plaque_likely_omits_it: z.string().nullable(),// short, honest, not snide
});
const GuidedLook = z.object({
guided_look_id: z.string(),
visit_id: z.string(),
photograph_uri: z.string(), // gs:// URI of the visitor's photo
language_bcp47: z.string(), // "ta-IN", "sw", "en-GB", "es-MX"
attribution: Attribution,
identification_confidence: z.number().min(0).max(1),
identification_alternates: z.array(z.object({
title: z.string(),
artist: z.string(),
why_considered: z.string(),
})),
script_total_duration_seconds: z.number().min(60).max(120),
beats: z.array(ScriptBeat), // exactly 6 in order
omitted_context_fact: OmittedContextFact,
sources: z.array(Source),
disagreements: z.array(DisagreementNote),
museum_label_current_text_quoted: z.string().nullable(),
museum_label_url: z.string().nullable(),
visitor_language_tts_voice_name: z.string().nullable(),
audio_uri: z.string().nullable(), // populated after TTS call
reading_confidence_overall: z.number().min(0).max(1),
flagged_for_visitor_review: z.array(z.object({
field_path: z.string(),
reason: z.string(),
})),
});
type GuidedLook = z.infer;
```
### Common failure modes (and how to avoid them)
- Agent silently downgrades `thinkingLevel` on the the identification call call to save quota — pin `gemini-3.5-flash` with the matrix-specified `thinkingLevel` explicitly. Flash misidentifies works at a much higher rate; the whole product collapses if the wrong painting is named.
- Agent combines `google_search` grounding and `responseSchema` in one call — the SDK will error. Split: the grounded call emits JSON in the text body and is parsed server-side; the structured script-generation call runs without grounding, fed the prior call's grounded text as input.
- Model invents a piece of provenance not in any cited source — hard-block in the system instruction: the `OmittedContextFact` schema requires a non-null `source_id`. The server-side validator rejects any beat whose `sources_used_ids` reference no real source.
- Model produces a beat that exceeds its time budget (a 60-second "opening orientation") — clamp on the server using `approx_duration_seconds`. If any beat exceeds its budget by >50%, re-prompt with the offending beat and the duration constraint reiterated.
- Region cue bounding boxes drift off the visible work (the script says "the dog" but the bbox is on a wall) — validate `bbox_normalised` is inside [0,1] on the server. If region cues are clearly wrong (no visible content), drop the region overlay rather than draw it incorrectly. The voice still speaks the cue.
- Identification confidence is high but obviously wrong (visitor at MUAC, model returns "Mona Lisa") — surface a "tell us which one" confirmation step whenever confidence is below 0.85, and whenever the model's location guess for `current_location.city` differs from a coarse device-locale hint, ask the visitor.
- TTS reads non-Latin scripts in a Latin-default voice — pin the TTS voice by `languageCode` matching the script language. Tamil text rendered by an English voice is unintelligible.
- Multilingual switching mid-script — Gemini 2.5 TTS cannot switch voices mid-call. Either keep the whole script in one language with the artist's foreign-language title in italics on screen, or stitch two TTS calls client-side.
- Plaque OCR misread on a work whose label is in a non-Latin script (Amharic, Khmer, Tamil) — Gemini 3.5 Flash multimodal handles these, but only if the system instruction tells it the script may be in a non-Latin alphabet. Do not assume Latin script.
- Sources listed in `Source[]` but not actually used by any beat — server-side validator: every source must be referenced by at least one beat's `sources_used_ids`. Prune unused sources before saving.
- Model writes the script in the visitor's language but quotes the museum label in the original (which the visitor cannot read) — the system instruction must say: quote the museum label verbatim AND include a short translation immediately after, in the visitor's language.
### Negative constraints (hard rules)
- Do NOT invent provenance. Every provenance claim must tie to a specific source. If no source exists for a provenance period the visitor asks about, say so plainly.
- Do NOT invent biography about the sitter, the patron, or the artist. The script may say "the sitter is generally identified as X" with a source. It may not say "X had a difficult relationship with Y" without a source.
- Do NOT pick a side in a public scholarly disagreement. Surface both positions with sources and let the visitor read on if they want. If the museum's current label takes a position, quote that position.
- Do NOT use the older label when the museum has replaced it. If the wall plaque the visitor photographed is out of date relative to the museum's online catalogue, prefer the online catalogue and surface the change.
- Do NOT produce a script that exceeds 110 seconds in any language. Even languages with longer word counts must fit.
- Do NOT use the visitor's photograph or any of their captured data to train or fine-tune any model. Use the Gemini API on the paid tier, where Google does not use your content for model training, per the Gemini API Additional Terms.
- Do NOT make a "fun fact" up to fill the omitted-context-fact slot. If nothing notable can be said about this object beyond what the plaque already says, the slot stays empty and the script is shorter. Honesty over completion.
- Do NOT translate the artist's name or the work's title. Quote them in the museum's labelled form, then add a short gloss in the visitor's language on first appearance only.
- Do NOT decorate the script with adjectives ("masterful", "stunning", "iconic"). The visitor is standing in front of the work; the work is doing the work. The voice points; it does not gush.
- Do NOT auto-share, auto-post, or surface the visitor's archive to anyone else. Sharing is explicit, per look.
### Per-call `systemInstruction` strings
Use these as the literal `systemInstruction` field for each Gemini API call the built app makes. They complement the series-wide rules already uploaded as the global instructions file (`00-series-instructions.txt`).
### Call: Identify the work from photograph(s) + plaque OCR + grounded research
Model: `gemini-3.5-flash` · thinkingLevel: medium · Tools: `google_search`
```
You are identifying one specific museum object from a visitor's
phone photograph and producing the grounded research the next call
will turn into a 90-second guided look.
Input: one or two images from the visitor's phone. Image 1 is the
object (a painting, a sculpture, a manuscript page in a vitrine, an
icon, a scroll, a ceramic, a printed image, a textile). Image 2, if
present, is the wall plaque or label beside the object. The visitor
may also have typed an artist + title to skip identification — if
they did, you receive that as text and the photographs serve only
as confirmation.
The photograph is taken under museum lighting: mixed warm tungsten,
indirect daylight from skylights, sometimes overhead halogen,
sometimes a vitrine reflection or glass-on-painting glare. The work
may be foreshortened, partially obscured by another visitor, or shot
from an angle. Read what you can; do not guess what you cannot read.
The wall plaque may be in any language. Common languages on museum
plaques include English, French, Spanish, Italian, German, Dutch,
Russian, Polish, Arabic, Hebrew, Modern Greek, Mandarin (Traditional
or Simplified), Japanese, Korean, Vietnamese, Tagalog, Indonesian,
Malay, Thai, Khmer, Hindi in Devanagari, Tamil, Bengali, Punjabi in
Gurmukhi, Urdu in Nastaliq, Farsi in Nastaliq, Turkish, Amharic in
Ge'ez script, Swahili, Yoruba, Portuguese, and English. Do not
assume Latin script.
Your task:
1. Identify the object as precisely as you can: artist, title, date,
medium, dimensions, current institution.
2. Use `google_search` grounding to find:
- the museum's current online label / catalogue entry for this
work (highest priority);
- the catalogue raisonné entry if one exists on the open web;
- the most-cited recent scholarship (≤ 10 years) on this work;
- any public museum statement about a label change, attribution
change, restitution, or conservation in the last decade.
3. Collect 4–10 sources. Note which is the museum's primary source.
4. Read every source carefully. Identify the single most surprising
thing the museum's current wall plaque is unlikely to say.
Output JSON in the response text body (do NOT use responseSchema —
this call has grounding enabled and the two cannot be combined).
The JSON shape:
{
"attribution": { …Attribution fields, populated from the museum's
current label form, not your guess at canonical },
"identification_confidence": 0..1,
"identification_alternates": [ { title, artist, why_considered } ],
"museum_label_current_text_quoted": "...", // exact, verbatim
"museum_label_url": "...",
"research_extracts": [
{ "source_id": "S1",
"publisher_or_institution": "...",
"title": "...",
"url": "...",
"is_museum_primary_source": true|false,
"extracts": [ "...short verbatim quotes used for one or more
specific spoken claims, each ≤ 60 words..." ]
}
],
"omitted_context_fact": {
"category": "...",
"one_sentence_fact": "...",
"source_id": "S?",
"why_plaque_likely_omits_it": "..." // honest, not snide
},
"disagreements": [ {
"topic": "...",
"position_a_summary": "...",
"position_a_source_id": "...",
"position_b_summary": "...",
"position_b_source_id": "...",
"museum_current_position_summary": "..."|null
} ]
}
Server-side: read citation URLs from
`response.groundingMetadata.groundingChunks[].web.uri` — do NOT
trust URLs the model writes inside the JSON body; it will
hallucinate them. Cross-reference each source_id to a real chunk
URL before saving.
Hard rules:
- If you cannot identify the object with confidence ≥ 0.85, set
identification_confidence honestly and populate
identification_alternates with the candidates you considered.
The app will surface a confirmation step to the visitor.
- Never invent a museum label. If you cannot find the museum's
current online label, leave museum_label_current_text_quoted
null and say so in flagged_for_visitor_review.
- Quote the museum's labelled artist + title forms verbatim.
Common-name forms ("Velázquez", "Caravaggio") go in
artist_canonical only.
- omitted_context_fact MUST have a source_id pointing to a real
source. If you cannot find a fact the plaque omits that is
supported by a source, leave omitted_context_fact null. Do not
invent. The next call will compress the script accordingly.
- For works with public attribution disagreement (sitter identity,
workshop participation, date range), include both positions in
disagreements[]. Do not pick a side.
- If the museum has updated this work's label in the last decade
(provenance research, restitution disclosure, attribution
change), surface that in disagreements[] with the prior position
and current position both quoted.
No commentary outside the JSON.
```
---
### Call: Generate the 90-second guided-look script → `GuidedLook` schema
Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: (none) · responseSchema: GuidedLook
```
You write the spoken script for a 90-second guided look at one
museum object. You receive: the visitor's photograph URI, the
visitor's chosen language (BCP-47), and the grounded research
extracts produced by the prior identification call.
You are not a docent. You are not a tour. You are a friend who
loves this object and has 90 seconds with the visitor before they
walk to the next room. You point. You name the one thing the
plaque does not say. You stop.
Structure (exactly six beats, in order):
1. opening_orientation — 10s
One sentence: "You are looking at [artist], [title], [date],
[medium]." Then one short sentence orienting the visitor to
the work's overall composition. Do not summarise the whole
plaque.
2. eyes_first — 15s
The single place the visitor's eyes are probably already
resting. Acknowledge it. Tell them to stay there for a moment.
Name what they are looking at and one thing about it (a
gesture, a brushstroke, a colour choice) that tells them where
the meaning of the work starts. Include one region_cue with
a bbox_normalised tightly around that region.
3. eyes_next — 20s
Where the work asks the eye to go next. This is composition:
a diagonal, a sightline, a contrast of light. Name the
destination. Then name what they see there — and what it
means. Include one or two region_cues.
4. omitted_context_fact — 25s
The one fact the museum's current wall plaque does not say,
as supplied by the prior call. Spoken in the visitor's
language. Sourced. If the prior call left the fact null,
reduce this beat to 5 seconds — a single sentence about what
we do know — and add the saved seconds to the closing beat.
5. corner_detail — 15s
The small thing in the corner, the margin, the underdrawing,
the dog. Name it. Tell the visitor to look at it. Tell them
what changes when they see it. Include a region_cue tightly
around it.
6. closing — 5s
One sentence that the visitor can carry to the next object.
Never a summary of what you just said. Often a question they
could think about while they walk: "Look at how the next
painter in this room handles light from a window."
Hard rules:
- Write in the visitor's language_bcp47. Match the dialect and
formality of museum-going speech in that language. (Spanish
for Mexico City reads differently from Spanish for Madrid;
Mandarin for Taipei uses Traditional characters and Taiwanese
conventions; Vietnamese uses formal "quý khách" register at
the opening.) Use the visitor's language consistently — the
one exception is the artist's name and the work's title, which
stay in the museum's labelled form. Add a one-time gloss in
the visitor's language in parentheses on first occurrence:
"Las Meninas (சபை மங்கைகள்)" — never thereafter.
- Quote the museum's current label verbatim once, when
introducing the work, then translate it inline for the visitor.
- Every claim in every beat must trace back to a source_id from
the grounded research extracts. Populate
sources_used_ids[] for every beat. If you cannot source a
claim, do not make it.
- Stay inside the per-beat duration. Speech rate at ~150 words
per minute for the visitor's language (adjust for languages
with different syllable density; Mandarin reads faster per
word than English, Tamil reads slower; the SDK will measure
the produced TTS).
- Insert TTS-friendly pacing cues already: short pauses as `…`,
paragraph or beat boundaries as a blank line plus an em-dash
`—`. Gemini 2.5 TTS does not support SSML pause tags.
- Total script_total_duration_seconds must be between 60 and
110. If the omitted_context_fact is null, the total will be
closer to 65–75; that is correct.
- Do not say "stunning", "masterful", "iconic", "beautiful",
"amazing", "incredible", "absolute genius", or equivalent
adjectives in any language. The visitor is in front of the
work. The work does the work.
- Do not editorialise about other works, other museums, or the
art market. Stay with this object.
- Region_cues' bbox_normalised: x, y, w, h all in [0,1] with
origin at the top-left of the photograph_uri. Bounding boxes
must lie inside the visible work, not on a wall or a frame.
Output: the GuidedLook JSON matching responseSchema. No
commentary outside the structured output.
```
---
### Call: Re-narrate a script into a different visitor language
Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: (none)
```
You receive an already-written GuidedLook script in one language
and a target language (BCP-47) the visitor has switched to. You
re-narrate the script into the target language while preserving
its structure (six beats, same beat_ids, same region_cues, same
sources_used_ids, same omitted_context_fact, same disagreements).
Hard rules:
- Translate the text of every beat into the target language at
museum-going register.
- Do NOT translate the artist's name or the work's title. Quote
in the museum's labelled form. Add a one-time gloss in the
target language in parentheses on first occurrence — never
thereafter.
- Preserve every quoted museum-label passage verbatim, then add
an inline translation in the target language.
- Preserve every region_cue's bbox_normalised, label_short, and
shape_hint unchanged. Translate spoken_phrase_verbatim only.
- Preserve every sources_used_ids and source_id reference
unchanged.
- Recompute approx_duration_seconds based on the translated
text. Total must remain between 60 and 110.
- Preserve the pacing cues (`…` for pauses, blank line plus `—`
for beat boundaries). Add or remove pauses if the target
language's natural rhythm requires it.
- Match the formality and dialect appropriate to the target
language's museum-going register, as in the script-generation
call.
Output: the updated GuidedLook JSON. No commentary.
```
---
### Call: Generate TTS audio of the script
Model: `gemini-3.1-flash-tts-preview` · n/a · n/a
```
Voice: a museum-floor voice — warm, unhurried, but slightly more
forward in pace than a reading-letters-at-the-kitchen-table voice
because the visitor is standing. Pick the Gemini 2.5 Flash TTS
voice whose `languageCode` matches the script's language_bcp47 —
pronunciation will follow that locale automatically. Prefer a
voice of any gender; the visitor's preference, if set, is in
`user.preferred_voice_gender`. Fall back to whichever voice is
available rather than blocking.
Pre-process the script before sending it to TTS:
- Concatenate the six beats' text in order, separated by a blank
line plus an em-dash `—` between beats. The blank-line+em-dash
produces a natural beat boundary pause.
- Within a beat, `…` already encodes short pauses; leave it
alone.
- Skip every region_cue label, every bbox, every sources_used_ids
reference — these are visual UI metadata, not spoken content.
- Target rate: ~135–155 words per minute depending on language —
museum-floor pace. The script-generation call already wrote to
this pace.
Style direction: prepend ONE short directive sentence to the
text input, exactly like: "Read warmly and clearly at a museum's
floor pace, as a friend pointing at a painting next to a visitor
who is standing.". Do not include this directive in subtitles.
There is no separate `style` API field on Gemini 2.5 TTS; the
directive sentence inside the input is how style is conveyed.
Mid-call voice switching is not supported. If the script contains
an artist's name or a title in another language, do not attempt
to switch voices. The voice will read the foreign-language string
in the chosen voice's locale; the on-screen transcript shows the
verbatim string italicised.
Phoneme overrides (Polish ł, German ü, Hebrew final letters,
Arabic emphatic consonants, Vietnamese tones, Mandarin tones,
Amharic ejectives) are NOT exposed by Gemini 2.5 TTS — no SSML
`` tag, no SSML `` tag, no separate `style`
field. Pronunciation comes from the chosen voice's native locale.
Cache key: `(work_id, language_bcp47, voice_name, script_hash)`.
A cache hit means we do not call TTS again — billing matters.
```
---
### Call: Long-description (accessibility) mode — extend script to ~180s
Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: (none)
```
You receive an existing 90-second GuidedLook and produce a longer,
~180-second version for visitors with low or no vision. The
extended version trades the "point with your eyes" structure for
denser spatial language and concrete description.
Hard rules:
- Preserve the six beat_ids. Extend their text. Do not invent new
beats.
- For each beat that previously instructed "look here", instead
describe what is there in vivid spatial language: position
(clock positions, left/right, foreground/background),
proportion, colour, brushwork or material, texture as visible
in the photograph, gesture or expression where applicable. Do
not invent material details the photograph cannot support — if
you cannot tell whether a surface is rough or smooth, say so.
- Keep the omitted_context_fact beat sourced. Length may grow but
the sourcing rule does not relax.
- Preserve region_cues but mark a new boolean
`is_used_for_audio_description: true` on every region_cue used
during the longer description — these guide stereo panning if
the device supports it.
- Total script_total_duration_seconds between 150 and 220.
- Same no-adjectives rule. The visitor is being shown the work
through language, not sold it.
Output: the updated GuidedLook JSON. No commentary.
```
---
### Call: Resolve disagreement between two label versions
Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: (none)
```
You receive two versions of a museum's label for one work — the
text the visitor photographed on the wall, and the text currently
published on the museum's online catalogue. The two differ in a
non-trivial way (attribution language, provenance statement,
sitter identification, date range).
Produce a one-paragraph note for the visitor explaining:
- exactly what changed between the two versions, in plain
language;
- which version the museum currently considers authoritative
(default: the online catalogue);
- whether the change is part of the museum's published label-
update programme (provenance research, restitution work,
rebrowsing of attribution) — only if a source supports this.
Hard rules:
- Quote both versions verbatim before paraphrasing.
- Do not editorialise about the change ("a long-overdue
correction", "a controversial decision"). State what changed.
- If the prior research extracts include a museum statement
about the change, cite it.
Output: a single string, no JSON, no commentary outside the
paragraph.
```
## 5. Use cases & content to include
Build dedicated UI sections or flows for each of these — they tell you what content the app must support.
- **The processional cross in Addis Ababa.** A visitor at the National Museum of Ethiopia stands in front of a thirteenth-century Lalibela-style processional cross. The wall card is bilingual Amharic + English but stops at "13th century, Lasta region". The app identifies the cross, surfaces three sources including the museum's own online catalogue and a recent paper on Ethiopian processional crosses, and the omitted-context fact is the iconography of the four corners — each one represents an Evangelist via a non-figural symbol the visitor would never decode on their own.
- **The Northern Song landscape in Taipei.** A visitor at the National Palace Museum stands in front of a hanging-scroll landscape. The visitor's eye does not know where to begin in the density. The app identifies the work, the script names "you are probably looking at the dark pine mass in the lower third — stay there", then walks the eye up the implied path of a traveller climbing the diagonal, then to the small figures at the pavilion, then to the calligraphy in the upper right which (the omitted-context fact) is a poem added by a Qing emperor four hundred years after the painting was made.
- **The Frida Kahlo at MUAC.** A visitor at the Museo Universitario Arte Contemporáneo in Mexico City stands in front of a Kahlo work on loan. The visitor speaks Spanish; the museum label is in Spanish; the visitor still wants the guided look because the show is dense and they have twenty minutes. The app speaks in Mexican Spanish at the right register, names what the museum's loan label adds beyond a typical Kahlo wall card, and the corner-detail beat points at the specific iconographic element the visitor would otherwise miss.
- **The Velázquez in Madrid.** A visitor at the Prado stands in front of Las Meninas. The app does not retell the famous mystery; it names what current scholarship has settled (the sitters, the date, the X-ray underdrawing), and the omitted-context fact is a recent conservation finding the wall plaque was not updated to include. The app surfaces the museum's online catalogue page as the primary source.
- **The plaque-changed work at the British Museum.** A visitor stands in front of a Benin bronze. The wall plaque has been replaced in the last decade to address its acquisition history. The app surfaces both the prior label and the current label verbatim, quotes the museum's published statement about the change, and does not editorialise. The visitor reads both and decides what to think.
- **The contested attribution at the Rijksmuseum.** A visitor stands in front of a small genre painting. Recent scholarship has split between Rembrandt and his workshop. The app does not pick a side. It surfaces both positions, names the leading proponent of each, and notes the museum's current label position. The visitor learns something the plaque does not have room to say.
- **The accessibility long-description.** A low-vision visitor at the Met stands in front of a Sargent portrait. They tap "describe this for me carefully" and receive a 180-second extended audio description with dense spatial language — sitter's pose at 11 o'clock, dress fabric described as visibly satin from the highlight pattern, hands clasped at lap level, gaze directed left of frame — that lets them carry an image of the work away.
- **The Tagalog-speaking visitor at the Louvre.** A Filipino visitor on a long-anticipated trip stands in front of the Winged Victory of Samothrace. The wall labels are in French and English. The visitor sets the app to Tagalog. The script speaks Filipino register, quotes the museum label verbatim with a Tagalog gloss, and the omitted-context fact concerns the figure's lost arms and how the museum displays the conservation history.
- **The pair of friends.** Two friends visit the Kunsthistorisches Museum together. They are at different stages of their German — one fluent, one beginning. They use the "pair phones" feature: the first phone hears the script in German, the second in English. The phones stay synced so the friends listen at the same pace and can talk between objects.
- **The art-history student.** A student preparing for an exam stands in front of a Caravaggio. They already know the canonical reading. They tap "give me the contested reading" and the app produces a script that emphasises the public disagreements in the scholarship rather than the museum's settled label.
- **The school group leader.** A teacher with twelve thirteen-year-olds in front of a sarcophagus at the Cairo museum needs a single object held for two minutes. The app produces a script the teacher can listen to silently first, then narrate to the group in their own words. The "save the look" archive becomes a record of the lesson.
- **The unattributed work.** A visitor stands in front of a work whose plaque reads "Spanish school, c. 1700, artist unknown". The app does not invent an attribution. The script is shorter, the omitted-context fact is about the kind of workshop production this likely came from, and the visitor leaves the work understanding what is known and what is not.
## 6. Page structure
Build the following screens / sections in this order. Adjust copy to fit the voice, but keep the structural intent.
1. **Welcome / sign-in.** A photographed-looking image of a visitor's phone held at chest height in a quiet gallery, an out-of-focus painting behind it, the visitor's hand steady on the screen. One paragraph: "Museum Whisper gives you a ninety-second guided look at the painting in front of you — in your language, with the one piece of context the wall plaque left out." Single Google sign-in button; Apple sign-in next to it. Below: "Use without signing in" → enables an on-device session with no cross-device sync. Below that: "Try with the sample painting" → loads the demo painting in section 8a.
2. **Empty state — "Point your phone at the painting".** Big circular shutter button. Underneath: "Best if the work fills most of the frame, with a corner of the wall plaque if you can fit it". A small text-only fallback: "I already know what this is" → opens a search box for artist + title.
3. **Capture flow** (mobile-first). Live viewfinder with a soft rectangular crop guide. Single shutter capture; an optional "add a plaque shot" prompt immediately after, with a smaller plaque-shaped guide. Auto-focus, auto-exposure, holds white balance steady. The flow handles glass glare by inviting a second shot from a slightly different angle if the first detects strong specular reflection.
4. **Identification confirmation** (only when identification_confidence < 0.85, OR when identification_alternates is non-empty). Up to three candidate works rendered side-by-side as reference thumbnails (pulled from a public museum API when available; otherwise rendered as title + artist + date cards). The visitor taps the correct one, or "none of these — let me type it".
5. **Generating screen.** A calm progress sequence with named steps, not a spinner: "Reading the painting…" → "Looking up the museum's current label…" → "Reading recent scholarship…" → "Writing your ninety-second look in [language]…" → "Recording the audio…". Each step takes 4-10 seconds.
6. **Guided-look detail view.** The visitor's captured photograph fills the upper portion of the screen, slightly desaturated to keep the eye on the work. A play button overlays the photograph. The transcript scrolls below, beat-by-beat, with each beat's region cues marked tappable. Tap a beat → a soft coloured ring is drawn over the corresponding region on the photograph. Sticky header: artist · title · date · language picker · "(i) sources".
7. **Sources panel.** A sliding panel from the right (desktop) or bottom sheet (mobile) listing every Source[] entry with its publisher, title, URL, and the beats it supports. Each entry tappable to open in the visitor's browser. The museum's primary source is marked with a small badge.
8. **Disagreements panel.** Only present when disagreements[] is non-empty. Two side-by-side cards: position A, position B, each with its source and a one-line summary of the museum's current position. No CTAs, no "vote".
9. **"What the plaque does not say" highlight.** A small bookmark icon next to the omitted_context_fact beat; tapping it pins the fact in a discreet floating panel that stays visible as the visitor scrolls.
10. **Personal archive.** A grid of saved guided looks across all visits. Each tile shows the captured photograph, the museum, the date of visit, and the language. Tappable for full detail; long-press to rename or share.
11. **Pair-phone modal.** "Pair with a friend in this room". Generates a 6-digit code. Friend's phone enters the code and joins the same guided-look session, in their own chosen language. Audio plays in sync.
12. **Accessibility long-description toggle.** A persistent button in the detail view: "Describe this carefully for me (3 minutes)". Tapping it switches to the long-description script and re-renders TTS in the same language. The on-screen transcript becomes denser; region cues are repurposed for stereo audio panning where the device supports it.
13. **Footer.** "Made for one painting at a time." Privacy: "Your photographs stay yours." Capabilities `(i)` icon in header.
## 6b. First-visit onboarding
Show a **first-visit onboarding** the first time a visitor lands on the app (detect via `localStorage` flag; do not show on return visits). Three slides, dismissible at any time. Persistent re-entry: a `?` icon in the header reopens it.
**Slide 1 — What this is.**
- Headline: "Welcome to Museum Whisper."
- Subhead: "Stand in front of one painting and hear what to look at next — in any museum, any script, any room."
- One paragraph (≤ 60 words) explaining: this app gives you one carefully-chosen ninety-second guided look at a single object. It is not a tour. It identifies the work from your phone, looks up the museum's current label and recent scholarship, and tells you where to look next and one piece of context the wall plaque does not say. Every claim has a source.
- Visual: a small annotated illustration of a phone held up at a painting with a soft circular ring drawn over a small detail in the lower-left of the painting — the corner-detail cue.
**Slide 2 — Try it now.**
- One short prompt: "Try with the sample painting".
- A live demo input pre-loaded with the sample painting in section 8a.
- 1-2 sentences pointing at *the specific page elements* where the Gemini magic happens (the identification confidence, the omitted-context fact, the source panel with a real museum URL).
**Slide 3 — How to remix this.**
- Headline: "Make this yours."
- Three short bullets:
- "Swap the sample painting in `/data/seed-painting/` for your own museum's collection."
- "Adjust the prompts in `/server/prompts/` to fit your visitors' languages and reading levels."
- "Wire up your Gemini API key and Firebase project via the env-var list in the capabilities panel."
- Primary CTA: "Use this template" → links to AI Studio Build remix entry point.
- Secondary: "Just exploring — close" (sets localStorage flag, never auto-shows again).
**Accessibility:** focus trap, `Esc` closes, `role="dialog"`, `aria-modal="true"`, `aria-labelledby`, focus restored to trigger on close. Respect `prefers-reduced-motion`.
**Don't:**
- Don't gate content behind the modal. The page beneath must be fully usable.
- Don't auto-reshow on return visits. Use `localStorage['onboarding-seen-v1']`.
- Don't include unrelated CTAs (newsletter signup, social follow). Keep it about the template only.
## 6c. Capabilities info button (persistent in header)
Add a persistent `(i)` icon in the top-right of the header (next to the primary nav). Click → opens a modal/panel titled **"What powers this app"**.
**Panel contents (in this order):**
**Gemini capabilities used (the hero list):**
- **Gemini 3.5 Flash (multimodal)** — identifies the work from a phone photograph taken under museum lighting, reading the work itself and the wall plaque if visible. One call per look; up to two images in one call.
- **Gemini 3.5 Flash (grounded search)** — fetches the museum's current online label, the catalogue raisonné entry where on the open web, and recent scholarship URLs. Citations come from grounding metadata, never from the model's free-text body.
- **Gemini 3.5 Flash (multilingual)** — writes the 90-second script in the visitor's chosen language with museum-going register. Tagalog, Vietnamese, Mandarin (Traditional and Simplified), Cantonese, Korean, Tamil, Hindi, Urdu, Bengali, Punjabi, Amharic, Swahili, Farsi, Khmer, Arabic, Hebrew, Greek, Polish, German, French, Italian, Spanish, Portuguese, English, Quechua, and more.
- **Gemini 3.5 Flash** — re-narrates a script into a different language when the visitor switches mid-visit, and resolves disagreements between a wall plaque and the museum's online label.
- **Gemini 2.5 Flash TTS** — narrates the script in the visitor's chosen language at museum-floor pace. Voice is picked by `languageCode`; pronunciation follows the voice's native locale. No SSML support — pacing is encoded as `…` and blank-line `—` in the script itself.
- **Firebase Auth** — Google and Apple sign-in (optional; the app works without sign-in for on-device use).
- **Firestore** — stores your saved guided looks and syncs across devices when signed in.
- **Firebase Storage** — keeps your captured photographs and generated audio at full resolution, privately.
- **Cost note** — see the detailed breakdown in 6d. A typical guided look costs about $0.06 of Gemini API spend. A day at a museum with twelve looks costs less than a coffee.
- **Privacy note** — your captured photographs are private to you. This app uses the Gemini API on the paid tier, where Google does not use your content for model training, per the Gemini API Additional Terms. The app does not report your visit to the museum, to Google, or to anyone else.
**Backend services this app depends on:**
- Auth: see section 4b (optional)
- Database: see section 4b
- Storage: see section 4b (must be enabled in Firebase console — not auto-provisioned)
- Email: not used in v1
- Payments: not used in v1
- External APIs: see section 4b (only the Gemini API; public museum APIs are optional)
**Environment variables you'll need to configure:**
- `GEMINI_API_KEY` — your Google AI Studio API key
- `FIREBASE_PROJECT_ID` — your Firebase project id
- `FIREBASE_SERVICE_ACCOUNT` — service-account JSON (server-side only)
- `FIREBASE_STORAGE_BUCKET` — your Firebase Storage bucket (after you enable it in the Firebase console)
**Cost + privacy notes:**
- One short paragraph per cost-sensitive capability: the grounded identification call is the most expensive call in the app (~$0.05); the script-generation call is small (~$0.01); TTS is small (~$0.005); the per-language re-narration is small (~$0.002).
- One short paragraph on privacy: where the data lives (your Firebase project), how to delete it (Settings → "Delete this visit" or "Delete all my saved looks" — gone in 60 seconds), what is never sent for training.
**Documentation links:**
- AI Studio Build docs
- Gemini API multimodal, grounded search, multilingual, TTS docs
- Firebase Auth, Firestore, Firebase Storage docs
**Accessibility:** same standards as the onboarding modal — focus trap, `Esc`, ARIA, restored focus.
**Behaviour:**
- Always available — single click from anywhere in the app.
- Tooltip on the `(i)` icon: "How this app is built".
- Mobile: opens as a full-screen sheet that slides up.
- Should be the most honest part of the app — never hand-wave service requirements; never say "AI" without naming the specific Gemini model and capability.
## 6d. Detailed cost breakdown (deployer reads this BEFORE shipping)
- **Identify the work + grounded research (Gemini 3.5 Flash, medium thinking, with `google_search`)** — typical 2 images (work + plaque), the call processes 4-10 grounded source extracts and returns ~1.5K output tokens. **~$0.045/look.** This is the dominant cost.
- **Generate the 90-second guided-look script (Gemini 3.5 Flash, low thinking, structured output)** — input is the research extracts and the visitor's language; output is ~1.2K tokens of structured JSON. **~$0.008/look.**
- **Re-narrate into a different language (Gemini 3.5 Flash, low thinking)** — input is the existing script (~1.2K tokens), output is ~1.2K tokens. **~$0.002 per additional language.** Cached per `(work, language)` thereafter.
- **TTS narration (Gemini 2.5 Flash TTS)** — billed per output token (~$10/M output tokens), effectively ~$0.000003/character. A 90-second script averages ~1,400 characters in English, longer in Tamil or Amharic. **~$0.004–$0.007/look.** Cached per `(work, language, voice, script_hash)`; charged once.
- **Long-description (accessibility) mode (Gemini 3.5 Flash, low thinking)** — extends the script to ~180s; same input-cost shape as script generation. **~$0.012/look** when invoked.
- **Disagreement resolution (Gemini 3.5 Flash, low thinking)** — only fires when the wall plaque differs from the museum's online label; rare. **~$0.001/look.**
- **Image storage:** Firebase Storage standard tier, ~$0.026/GB/month. A typical captured photograph is ~2-3 MB. A visitor with 100 saved looks uses ~0.3 GB ≈ ~$0.008/month.
- **Audio storage:** ~$0.026/GB/month. A 90-second MP3 at 64 kbps ≈ ~700 KB. 100 saved looks ≈ ~70 MB ≈ pennies/month.
- **Expected per-look cost on first generation:** ~$0.06. **A twelve-look day at a museum:** ~$0.72. **One additional language switch per look:** ~$0.006 each.
## 7. Design language
- **Mood:** A quiet gallery on a weekday afternoon. The room is half-lit. The visitor is standing. The app is the voice in their headphone, not the wallpaper of the room. It is never a museum kiosk; it is never a tech product; it is never a tour bus screen.
- **Typography:** Display serif for object titles and beat headings (Source Serif Pro or Adobe Caslon Pro), at a size that respects the work's own type-treatment on its wall plaque. Clean grotesque for app chrome (Inter or Geist). Body for the transcript in a comfortable serif at ~17–18 px on mobile, ~19 px on desktop. Each beat's heading sits a hair larger than the body — recognisable as a heading, not a tab.
- **Palette:** A near-black `#141414` background for the photograph-foregrounded views (visually a dim gallery wall behind the work). A warm bone `#F4EFE6` for the transcript surface (paper-feel, calm). Deep ink `#1B1714` for body text on bone. A muted gold `#9C7B3E` for the omitted-context-fact panel border and for the small "saved" check (museum brass, not luxury gold). A muted teal `#2F605C` for region-cue rings (visible on most paintings without competing with the work). A soft red `#A33A2C` reserved for the rare disagreement chip. Avoid SaaS blue. Borrowed from gallery walls and conservation labels, not from product design systems.
- **Imagery:** The visitor's captured photograph is the hero. Never replace it with a stock catalogue image of the same work; the visitor's perspective is what matters. The photograph sits with breathing room around it; never bleeds edge to edge except on the focused detail view.
- **Hand-feel touches:** Region-cue rings draw on the photograph with a slow stroke (200 ms) the first time, then sit static. Tapping a beat in the transcript gently scrolls the photograph if the cue is outside the current viewport. The pair-phones modal's 6-digit code is set in a generous monospace, comfortable for the visitor in low gallery light.
- **Spacing:** consistent 4-px base. Generous whitespace — gallery air, not dashboard density.
- **Radius:** consistent token set (e.g. 6 / 12 / 20 px). Thumbnails use 6; cards use 12; modal sheets use 20.
- **Shadows:** subtle, layered, warm. The photograph view uses a single soft shadow under the photograph so it floats on the dim wall background. Otherwise restrained.
- **Motion:** purposeful — beat transitions on tap, the soft region-cue stroke, the pull-up of the bottom sheet on mobile. Respect `prefers-reduced-motion`: the region-cue stroke draws instantly, the photograph-shift on beat-tap jumps rather than animates. Audio playback is never accompanied by a moving equaliser or waveform; it is a single calm playhead bar at the bottom of the transcript.
- **States:** every interactive element has hover, focus, active, disabled. Loading uses named-step text not spinners. The capture viewfinder has a steady, non-distracting frame guide that never blinks.
## 8. Content generation rules
- Write **realistic, specific copy**. NO Lorem Ipsum. NO generic placeholders like 'Your tagline here'.
- Invent plausible museums, paintings, attributions, and sources that fit the domain (use the seed content in section 8a as a starting point). When inventing demo content, use real-sounding but fictional sources — never claim a fictional URL belongs to a real museum, and never reproduce a real museum label's text. The seed content in 8a uses generic descriptions in place of real label text.
- Tone: warm, direct, free of corporate language. This template is for a person, not a company.
- Headlines: punchy and concrete. No 'Empower your X' filler. No 'Revolutionize'. No 'Seamless'.
- Body copy: short paragraphs (2-4 sentences). Use lists where appropriate.
- Plain language. Avoid jargon — except where the user already speaks the jargon (the art-history-student user wants to see "catalogue raisonné" and "attribution percentage"; the docent-trainee wants to see "iconography" and "underdrawing").
- Where the app outputs AI-generated content, never label it as "AI says" — let it speak naturally. Use small uncertainty cues only where epistemic honesty requires them (a low-confidence identification surfaces alternates; a sourced claim shows its source on tap; an unsourced claim does not exist).
## 8a. Seed content (use these specific examples)
Anchor every generated copy + sample data point in the concrete content below. Use these names, numbers, dates, and snippets verbatim where helpful, or generate close variants that sit in the same world. All museum names below are real; all label text and source URLs in the sample painting are illustrative only — the built app produces these from real grounded search at runtime.
**Sample museums (sidebar — the visitor's "where am I" picker):**
- National Museum of Ethiopia (Addis Ababa)
- National Palace Museum (Taipei)
- Museo Universitario Arte Contemporáneo / MUAC (Mexico City)
- Museo del Prado (Madrid)
- The British Museum (London)
- The Rijksmuseum (Amsterdam)
- The Metropolitan Museum of Art (New York)
- The Louvre (Paris)
- Kunsthistorisches Museum (Vienna)
- The Egyptian Museum (Cairo)
- The Iziko South African Museum (Cape Town)
- The Asian Art Museum (San Francisco)
- The National Museum of Korea (Seoul)
- The National Museum, New Delhi
- Bảo tàng Mỹ thuật Việt Nam / Vietnam Fine Arts Museum (Hanoi)
**Sample painting in detail view (this is what the demo should show):**
- **Captured photograph:** a phone shot of a small oil-on-panel portrait in a wooden frame, lit by indirect gallery daylight, no glass glare, with a corner of the wall plaque visible in the lower-right.
- **Attribution as labelled (verbatim):** "Attributed to the workshop of an unnamed Sevillian master, c. 1635"
- **Title as labelled:** "Portrait of a Woman with a Lapdog"
- **Date as labelled:** "c. 1635"
- **Medium as labelled:** "Oil on panel"
- **Dimensions as labelled:** "47.2 × 38.5 cm"
- **Current location:** Sample Museum of Iberian Painting, Room 12 (fictional museum for demo only)
- **Attribution confidence (scholarly):** "workshop_or_circle"
- **Identification confidence (model):** 0.91
- **Identification alternates (1):** one alternate considered — a smaller workshop portrait of similar composition from the same room (why_considered: "similar frame, similar sitter pose, same period")
- **Museum label current text quoted (illustrative):** "Attributed to the workshop of an unnamed Sevillian master, c. 1635. Oil on panel, 47.2 × 38.5 cm. The sitter has not been securely identified; a recent campaign of provenance research has updated this label to reflect current scholarship."
- **Beats (6, illustrative):**
- `opening_orientation` (10s): "You are looking at a small Sevillian portrait from around 1635 — a woman seated three-quarter length, a small dog on her lap. The work is on oil panel, attributed to the workshop of an unnamed master from Seville."
- `eyes_first` (15s): "Your eyes are probably on her face — stay there for a moment. The light enters from the upper left and rakes across her right cheek. The painter has built the highlight in one confident stroke; the shadow on the left side of her face is layered in glaze."
- `eyes_next` (20s): "Now drop down and to the left, to the dog. Look at the dog's eye. It is the only place in the painting where the sitter is being looked back at — she is not looking at us, the dog is. In seventeenth-century Iberian portrait practice this gaze-direction is often a marker of household intimacy: a lapdog is a domestic witness, painted from life from the household's own animal, not from a pattern book."
- `omitted_context_fact` (25s): "The plaque tells you the sitter has not been securely identified. What it does not tell you is that recent dendrochronology on the panel dated the wood to the same Seville workshop that produced two other documented portraits of the Saavedra family between 1632 and 1638. The sitter is now thought, by a minority of scholars, to be one of those women. The museum has not adopted the identification, but the panel itself argues for the workshop."
- `corner_detail` (15s): "Look at the lower-left corner. There is a small ribbon tied around the dog's collar. Its colour was once a deeper red; it has faded under varnish. Read more closely — the ribbon's tie is a knot known in seventeenth-century Spanish needlework as the 'lazo de novia'. It is a wedding-marker."
- `closing` (5s): "Walk to the next portrait in this room. Look at its sitter's hands."
- **Omitted-context fact (sourced):** category "scholarly_disagreement"; one-sentence fact as above; source_id "S2" (the dendrochronology paper); why_plaque_likely_omits_it: "the museum's policy is to update labels only after a finding is supported by majority scholarly consensus, not a single paper."
- **Sources (4, illustrative):**
- S1 — The museum's own online catalogue entry for this work (museum primary source). Used for: attribution, dimensions, current label.
- S2 — Open-access paper on dendrochronology of Sevillian seventeenth-century panels. Used for: omitted-context fact.
- S3 — Catalogue raisonné chapter on Sevillian workshop portraits. Used for: eyes_next interpretation.
- S4 — Open-access study on lazo de novia symbolism in Iberian textiles. Used for: corner_detail interpretation.
- **Disagreements (1, illustrative):** topic "identity of sitter"; position A "unidentified" (museum current position, S1); position B "one of the Saavedra women, on workshop-dendro evidence" (S2); museum current position summary: "treats identification as not yet supported by majority scholarship".
- **Reading confidence overall:** 0.89
- **Visitor language for the audio:** "es-MX" (Mexican Spanish)
- **TTS voice name:** an appropriate Gemini 2.5 Flash TTS voice keyed to `es-MX`.
**Sample input artefacts (for the build to demonstrate):**
- A phone-quality photograph of a small oil portrait under gallery lighting with a corner of the wall plaque visible.
- A phone-quality photograph of a Northern Song-style hanging scroll under softer overhead lighting, with no plaque visible.
- A phone-quality photograph of a thirteenth-century processional cross in a glass vitrine, with reflections suppressed to a manageable level — to demonstrate the two-shot mode and the glass-glare retake prompt.
- A phone-quality photograph of a Benin bronze with the wall plaque visible — to demonstrate the disagreement-between-labels surface.
**Sample voice copy:**
- Onboarding: "Stand in front of one painting. We'll tell you where to look next."
- Capture: "Frame the work and a corner of the plaque if you can fit it."
- Identification confirmation: "We're 91% sure this is the work below. Tap to confirm — or tell us which one."
- Generating: "Reading the painting…" / "Looking up the museum's current label…" / "Reading recent scholarship…" / "Writing your ninety-second look in [language]…" / "Recording the audio…"
- Empty archive: "This visit is waiting for its first painting. Point your phone at one."
- Error (couldn't identify): "We could not place this work confidently. Want to add a photograph of the wall plaque, or type the artist and title?"
- Save confirmation: "Saved to today's visit at [museum name]."
- Disagreement detected: "The wall plaque and the museum's online label differ. Tap to read both."
- Low confidence note: "Some of what we found is contested. The disagreements panel shows both readings."
**Sample pair-phone share message:**
- "I'm at [museum name] in Room [number] — pair my phone to hear the same painting in [your language]. Code: [6-digit code]."
## 9. Media & assets
- **Hero image (landing screen):** A photographed-looking shot of a hand holding a phone at chest height in a quiet gallery, an out-of-focus painting behind it, the visitor's wedding ring catching a soft daylight from a skylight. Generate via Nano Banana 2 with a prompt emphasising "gallery floor, indirect skylight, a phone screen lit warmly, a partially visible painting in the background — not specific to any real work, a generic gestural figure painting, no museum branding, no recognisable visitors".
- **App icon / wordmark:** Set in the display serif. Slightly worn paper texture behind it. No icon — just type.
- **Empty-state illustration:** A simple line drawing of a phone held up at a painting, with a soft circular ring on a corner detail. Hand-drawn aesthetic, not a flat icon.
- **Demo painting photographs:** Generated per the prompts in section 8a — Nano Banana 2 prompts that specifically request "phone-quality photograph, gallery interior lighting, no glass glare in the first sample, a corner of a wall plaque visible in the lower-right, the painting is a small fictional seventeenth-century Iberian portrait of a woman with a lapdog — generic period style, not a real recognisable work, no museum branding visible". Each demo painting should look photographed by a visitor, not rendered.
- **Generic gallery imagery:** A small set of background-room images used only on the landing and pair-phone screens. Never specific to a real museum's room.
- **Stock fallbacks:** If image generation fails, fall back to a photograph of a single bone-paper textured card from `/public/samples/sample-card.jpg`. Never to a "🖼" emoji.
- **Generated imagery:** prefer Nano Banana 2 over stock photography. Prompt for warmth, asymmetry, slight imperfection — avoid the glossy 'AI render' look.
- **Optimisation:** WebP/AVIF, `loading="lazy"`, explicit `width`/`height` to prevent layout shift. The visitor's own captured photograph is NOT re-encoded to WebP — preserve the original JPEG with EXIF stripped of GPS metadata before upload.
- **Icons:** `lucide-react` for UI. Use sparingly — never decorative-only.
### Build-time asset manifest (explicit specs)
Every image, illustration, and visual reference mentioned above must resolve to ONE of the three buckets below — runtime-generated, seed-shipped, or user-supplied. Do NOT ship `` tags whose `src` is not listed here. Do NOT depend on bare "section 8a prompts" without binding them to explicit paths and model IDs.
**Bucket 1 — Runtime-generated (Nano Banana Pro `gemini-3-pro-image` for hero/demo photographs; Nano Banana 2 `gemini-3.1-flash-image` for in-app illustrations and reference-conditioned variants).** Cached to Firebase Storage; served via signed URL. Every reference above to "Nano Banana 2" or "Nano Banana Pro" MUST be wired to one of these specific calls with an explicit model id:
- `/public/generated/hero.webp` (2400×1500, WebP) — model `gemini-3-pro-image` — uses the literal prompt described as "Hero image (landing screen)" above. Run once at build; commit a `/public/samples/hero-fallback.webp` (1600×1000) generated from the same prompt with `gemini-3.1-flash-image` so the page renders if quota is exhausted.
- `/public/generated/demo/{demo-slug}-{NN}.webp` (1600×1200, WebP) — model `gemini-3.1-flash-image` (reference-conditioned where the prior frame is passed as input) — one path per "Demo X" image referenced above. The slug derives from the seed example in section 8a; the NN index covers each frame in the demo sequence.
- `/public/generated/illustrations/{name}.webp` (1024×1024, WebP) — model `gemini-3.1-flash-image` — one path per named illustration above ("Empty-state illustration", "Recipe-card hero illustrations", "Curriculum picker imagery", "Period-style frames", etc.). Each illustration's prompt is the literal description above; ship a deterministic seed in the request so re-runs are reproducible.
**Bucket 2 — Seed assets shipped with the deliverable.** Every "Stock fallback" path referenced above (e.g. `/public/samples/sample-X.jpg`) is generated once via Nano Banana 2 (`gemini-3.1-flash-image`) at 1024×1024 WebP using the same prompt as its Bucket-1 counterpart, then committed to the repo so the page renders identically if Gemini quota is exhausted or the user is offline. Replace any `.jpg` extension above with `.webp` to match the optimisation rule. Also commit these empty-state seeds (1024×1024 WebP, single-stroke hand-drawn line, no colour fill):
- `/public/samples/empty-state-primary.webp` — line drawing of the app's primary empty surface (the named "Empty-state illustration" above), generated from that exact prompt.
- `/public/samples/empty-state-archive.webp` — line drawing of an empty saved/archive view, single-stroke outline.
- `/public/samples/empty-state-error.webp` — line drawing of a hand placing a single object aside with care, used when an AI call fails.
**Bucket 3 — User-supplied.** Uploads from the user's camera / file picker land at the Firebase Storage path conventional for this template (named in section 4b). The build ships with Bucket-1 + Bucket-2 only; no user-supplied images at first paint.
**Hard rules**
- Every `` tag MUST have a `src` that resolves to a path listed in Bucket 1, Bucket 2, or a Bucket 3 upload path. Anything else is a build error.
- No bare `image.jpg` / `hero.jpg` / `placeholder.png` references anywhere in the code.
- Model IDs: `gemini-3-pro-image` for hero-quality photographic generation; `gemini-3.1-flash-image` for in-app illustrations, reference-conditioned variants, empty-state seeds, and stock fallbacks. Never use a legacy model id (no `imagen-*`, no `gemini-1.5-*-image`).
- File format: WebP everywhere (AVIF acceptable where the target browsers support it). No `.jpg` / `.jpeg` / `.png` in `/public/samples/`.
## 10. Interactivity & states
- Every interactive element has hover, focus, active, and disabled states.
- Forms validate inline and show specific error messages (not "Invalid input").
- Loading states use named-step text not spinners — the visitor sees what is happening.
- Empty states explain the next action with a button whose label fits THIS app's domain: "Point your phone at the painting", "Add a plaque shot", "Switch language", "Pair with a friend" — never a generic "Add your first item".
- Smooth scroll for in-page anchors, including beat-to-photograph cue scrolling.
- All AI-generated content streams in token-by-token where supported — the visitor sees the script unfold beat-by-beat before the audio is ready, with a small "audio is being recorded" indicator at the bottom.
- If an AI call fails, show a calm, specific error ("We could not place this work — want to add a photograph of the wall plaque?") and offer retry.
- Low-confidence identifications surface an explicit confirmation flow rather than silently picking. The model's alternates are shown with reference imagery where available.
- The map of beats (small dots in the transcript header) is tappable — visitor can jump to any beat.
- Region cues draw on the captured photograph with a 200 ms soft stroke; `prefers-reduced-motion` jumps to the final state.
- Pair-phone audio uses WebRTC for low-latency sync; if WebRTC fails the apps fall back to a shared playhead in Firestore that updates every 500 ms.
- Audio playback exposes a clear scrub bar and a 15-second rewind button — the visitor often wants to hear the corner-detail cue twice.
## 11. Tech & responsive requirements
- **TTS markdown-stripping preprocessor:** before sending any user-authored markdown to `gemini-3.1-flash-tts-preview`, strip non-spoken markdown: `#`/`##`/`###` headings (keep the title text), `**bold**` (keep the inner text), `[label](url)` (keep `label`, drop URL), `` ``` `` fenced code blocks (skip entirely), `>` block-quote markers (keep the text), and `|` table pipes (read row-by-row as sentences). Insert `…` between sentences for a short pause and a blank line plus `—` between paragraphs for a long pause. The model does not understand markdown; raw markdown will be read aloud as literal characters ("asterisk asterisk").
- **File downloads on Safari / Firefox:** when offering local-disk save of any export (PDF, CSV, MP3, ZIP, JSON, image), fall back to `` with a blob URL — the File System Access API (`showSaveFilePicker()`) is Chromium-only. Detect with `'showSaveFilePicker' in window`; otherwise use the anchor-download path.
- **Stack:** React + TypeScript + Tailwind CSS. Functional components + hooks. Use Shadcn UI primitives where appropriate.
- **Build runtime:** AI Studio Build — full-stack with Cloud Run server-side functions. All Gemini API calls happen server-side; API key lives in Secrets Manager, never in client bundle.
- **Model selection:** explicitly pin `gemini-3.5-flash` for identification + grounded research and for script generation; `gemini-3.5-flash` for re-narration and disagreement resolution; `gemini-3.1-flash-tts-preview` for audio. Set `thinkingLevel` explicitly per call. Do NOT serialise `thinkingConfig` on TTS calls.
- **Image handling:** the visitor's photograph is uploaded server-side via the Gemini Files API (`files/*` resource name) before the identification call. Do NOT pass Firebase Storage public download URLs directly to `generateContent` — the Gemini API does not fetch them server-side.
- **Database:** Firestore (auto-provisioned by AI Studio Build). Show the seed painting on first launch.
- **Auth:** Firebase Auth — anonymous by default; Google sign-in optional; Apple sign-in optional and user-configured.
- **Storage:** Firebase Storage for captured photographs and TTS audio. Pre-signed URLs only. Storage NOT auto-provisioned — visitor must enable in Firebase console.
- **Mobile-first.** Verify layouts at 375 px (iPhone SE), 768 px (iPad), 1024 px, 1440 px+. The capture flow is mobile-only in v1; on desktop the empty state offers a file-upload alternative.
- Use `clamp()` for fluid typography. Prefer container queries over media queries for component-level responsiveness.
- Use `dvh` / `svh` instead of `vh`. Respect safe-area insets on iOS.
- Zero horizontal overflow at any width. Zero layout shift on load.
- Persist user data in Firestore. Use real-time listeners on the archive view and pair-phone session.
- Optimistic UI on writes; reconcile on response.
- Audio playback uses native `