================ ONE-SHOT BUILD CONTRACT (read first) ================
Build this in Google AI Studio "Build" in ONE shot — a complete, working app,
no follow-up turns. These are hard rules, not suggestions:
1. TARGET = Full-Stack Web (Node server runtime, secrets, Firebase allowed).
If you target Native Android instead, you MUST drop all server/DB/Workspace/
secrets and persist locally (Room / SharedPreferences) only.
2. PIN EVERY MODEL CALL — never let the agent auto-select (it downgrades on quota):
- Reasoning / text -> gemini-3.5-flash (thinkingLevel: minimal|low|medium|high)
- 4K image + legible text -> gemini-3-pro-image (image_size "4K", up to 14 refs)
- High-volume image -> gemini-3.1-flash-image
- Expressive TTS -> gemini-3.1-flash-tts-preview (inline tags e.g. [whispers])
- Realtime audio/video (WebSocket) -> gemini-3.1-flash-live-preview
- Sandboxed agent -> antigravity-preview-05-2026
3. DIVISION OF LABOR — the model ONLY parses/extracts to a strict responseSchema.
ALL math, money (store currency as integer minor units / cents), sorting,
balancing and graph logic run in deterministic TypeScript/Python. The model
must never compute totals, splits or balances itself.
4. responseSchema sanitation — no regex patterns, no fixed-length tuples, no
format validators in the schema (they crash the OpenAPI engine). Enforce those
in server-side code AFTER parsing the JSON.
5. responseSchema and google_search grounding are MUTUALLY EXCLUSIVE in one call.
6. CODEGEN — split large output into modular, single-responsibility files so no
file is truncated by the output-token cap.
7. Every external call gets a graceful fallback (e.g. manual paste if a Workspace
read fails). Never a silent dead end.
8. ROBUST STORAGE & CANVAS — Wrap all `localStorage`/`sessionStorage` operations (especially JSON parsing and writes) in `try-catch` blocks to prevent crashes in private windows or quota overflows. Canvas drawing elements must dynamically handle window resize and scale pixel density (`window.devicePixelRatio`) to avoid blurry graphics on retina displays.
=====================================================================
# MUST OBEY — Mobile-first build requirements
This app's PRIMARY surface is a mobile phone. Build it impeccably on mobile FIRST, then verify on tablet and desktop. Treat the rules below as non-negotiable hard constraints, not suggestions.
## Viewports to verify (every screen, every state)
- 320 px, 360 px, 375 px, 390 px, 414 px, 480 px
- 768 px, 834 px (iPad portrait / Pro 11)
- 1024 px, 1280 px, 1440 px, 1920 px, 2560 px
- Plus: 200% browser zoom, landscape orientation on every mobile width, iPhone with safe-area insets visible
## Hard layout rules
- Mobile-first CSS. Default styles target mobile; `@media (min-width: ...)` for larger viewports.
- Use `dvh` and `svh` instead of `vh` for full-height surfaces (iOS Safari URL-bar bug).
- Use `clamp()` for fluid typography across all viewports.
- Prefer container queries (`@container`) over media queries for component-level responsiveness.
- Use `min(100%, ...)` widths so content never overflows. Zero horizontal overflow at any viewport.
- Add `` to every page.
- Apply `padding: max(safe-area-inset-X, fallback)` on every edge-bleeding container so notched iPhones in landscape never clip content.
- Wide tables and code blocks scroll INSIDE their container (`overflow-x: auto`), never push the body.
- Use `background-attachment: scroll` on mobile, not `fixed` (iOS Safari repaint bug).
- Avoid `backdrop-filter` on animated elements. Use it sparingly on static surfaces only.
- **Canvas Scaling**: Canvases must dynamically scale with window resize events and properly handle high-DPI screens (`window.devicePixelRatio`). Set physical dimensions (`canvas.width`/`canvas.height`) using pixel ratio and render relative to this grid, using CSS to control responsive viewport scaling.
- **Robust Storage**: Every access to `localStorage`/`sessionStorage` (especially `JSON.parse` of loaded state or writes) MUST be wrapped in a `try-catch` block to handle disabled storage, private browsing mode, quota limits, or corrupted JSON gracefully. Fall back to a robust in-memory object store.
## Touch & accessibility
- Tap targets ≥ 44 × 44 px on touch (Apple HIG). Increase to 48 px under `@media (hover: none) and (pointer: coarse)`.
- All interactive controls reachable by keyboard with a visible focus ring; respect `:focus-visible`.
- Color contrast ≥ 4.5:1 for body text, 3:1 for UI components.
- All images have meaningful `alt`. Decorative images use `alt=""`.
- Respect `prefers-reduced-motion: reduce` — zero animation durations under that query.
- Forms validate inline; error messages are specific, not "Invalid input".
- Modals: focus trap, `Esc` closes, `role="dialog"`, `aria-modal="true"`, focus restored on close.
## Performance bar (Lighthouse mobile, throttled 3G/4G)
- LCP < 2.5 s · INP < 200 ms · CLS < 0.1
- JS bundle gzip < 200 KB mobile-first; lazy-load non-critical screens via `React.lazy` / dynamic imports.
- No render-blocking resources above the fold.
- Images: WebP/AVIF preferred, `loading="lazy"`, explicit `width`/`height` attributes (zero CLS), `srcset` for retina.
- Videos: `preload="metadata"`, low-resolution poster, max 720p mobile fallback. Never autoplay with audio.
- Fonts: `font-display: swap`; preload only the one used above the fold.
- Smooth scroll honoured via CSS `scroll-behavior: smooth` with reduced-motion fallback.
## Pre-ship mobile checklist (the deployer MUST verify before declaring done)
1. Open at 375 px in DevTools — every screen scrolls vertically only; zero horizontal scroll.
2. Browser zoom 200% — layout reflows without overlap.
3. iPhone Safari with the URL bar visible AND landscape — no content under the home indicator; no notch clipping.
4. iPad portrait (768 px) and landscape (1024 px) — no awkward gaps; tablet-specific breakpoints land cleanly.
5. Tap every interactive element with a thumb at real-device size — every target is easy to hit.
6. `prefers-reduced-motion: reduce` — every transition / animation skips cleanly, scroll-behavior becomes instant.
7. Lighthouse mobile score ≥ 90 across all 4 categories.
8. Zero `console.error` and zero CLS shift in real-device testing on a mid-tier Android (e.g. Pixel 6a) and an iPhone SE.
---
The original template starts below. All rules above apply on TOP of whatever this template specifies.
---
# Yarn Stash
## 1. Project
**Yarn Stash** is a personal yarn-inventory builder for the knitter,
crocheter, weaver, or felter who has accumulated more wool than she
can keep in her head. The user photographs her basket — or her
under-the-bed bin, or the IKEA tote behind the sofa, or the inherited
suitcase from her grandmother — and the app produces a labelled
inventory. Each ball or skein carries a thumbnail of the original
photograph, an estimated weight class, a best-guess fibre composition
with explicit confidence, a yardage estimate based on the visible
volume, a swatch-colour hex pulled from the photo, and a short list
of suggested projects calibrated to how much of that yarn is left.
This is the kind of app a Filipino-American knitter in San Jose
builds on a Saturday morning after the closet-rod finally gave up
under the weight of eighty unlabelled balls of yarn — half of them
inherited from her tita, half of them bought at Stitches West over
the past six years and shoved in plastic bags she never re-opened.
It is also the kind of app a Peruvian collector in Cusco builds for
her family's hand-spun alpaca stash going back three generations,
where each generation cared less and less about labels because each
generation knew exactly which sheep the fibre had come from. Same
shape of moment, different fibres, different hemispheres.
The single demo that proves the magic: photograph the open basket on
the floor → in under twenty seconds the app has tagged every visible
ball. Each one gets a card: "Aran weight (estimated), likely a wool /
alpaca blend (medium confidence — no label visible), approximately
180 metres remaining, good for a hat or a pair of mittens." The user
taps any card to refine it ("actually, I remember — that one's pure
merino, my mother brought it back from Arequipa"), and the inventory
updates instantly.
And in the harder cases — a half-burnt ball pulled out of a flood-
damaged box, a hand-spun single with no commercial reference,
mystery yarn donated to a community knit-night with the labels
already lost — the app refuses to bluff. It says "fibre uncertain"
loudly, lists the two or three possibilities, and asks the user to
do the burn test or the float test before committing the entry.
Estimates only, never definitive — the model never tells the user a
fibre composition with certainty when the label is missing.
**Tagline:** _Turn a basket of unlabelled yarn into a working
inventory — in any fibre, any weight, any colour, with every guess
held lightly and every original photograph preserved._
## 2. Target audience
- Hand-knitters with a stash that has outgrown their memory — six baskets, two bins, the bag under the bed, the shelf above the washer
- Crocheters who buy in mixed-lot bundles from estate sales and end up with twenty mystery skeins per haul
- Spinners and weavers cataloguing hand-spun singles, plied yarns, and one-of-a-kind dye-lots that no commercial reference will ever match
- Inheritors — the granddaughter who took home her grandmother's yarn cabinet, the daughter sorting her late mother's craft room, the niece who got the great-aunt's spinning wheel and three bins of unknown wool
- Diaspora makers cataloguing fibre brought back from home countries — Peruvian alpaca from Cusco, Mongolian cashmere from Ulaanbaatar, Shetland wool from Lerwick, Andean qiviut, Welsh Black Mountain, Filipino abacá blends, Ethiopian hand-spun cotton, Japanese sashiko thread, Indian khadi cotton, Bolivian llama
- Community-yarn-cabinet stewards — the librarian who runs the Tuesday-night knit-and-natter, the senior-centre activities coordinator, the prison-knitting-program volunteer, the school textiles teacher
- Indie-dyers cataloguing one-off skeins they dyed two years ago and forgot the recipe for
- Sensory-sensitive makers who can't tolerate certain fibres next to skin and need their stash sorted by what they can actually wear
- Sustainability-minded makers refusing to buy more yarn until they've used what they own — the "stash-down 2026" pledger
## 3. Core value propositions
Surface these clearly through copy, visual emphasis, and section ordering — they are the reasons users pick this app.
- **Reads any yarn, any fibre, any colour** — wool, alpaca, merino, mohair, cashmere, qiviut, angora, silk, cotton, linen, hemp, bamboo viscose, abacá, acrylic, nylon, polyester blends, hand-spun singles, novelty boucles, eyelash yarn. Gemini 3.5 Flash does the visual fibre estimation, the colour extraction, the weight-class call, and the yardage estimate in one call. The app never claims certainty it does not have.
- **Estimates are estimates** — every fibre guess carries an explicit confidence band and a one-line reason ("matte surface and long staple suggest a wool / alpaca blend, but a burn test would confirm"). The app surfaces the test the user could do herself rather than pretending to know.
- **The original photograph is the source of truth** — the photograph of the ball is always one tap away. The estimated metadata sits beside it, never replaces it.
- **Yardage estimates ground in physics, not vibes** — the volume estimate uses the visible diameter and length of the ball, the weight-class assumption, and a published yardage-per-gram table for the assumed fibre. The math is shown to the user, not hidden behind a magic number.
- **Suggested projects respect what is actually there** — a 40-metre scrap suggests a coffee-cosy or a doll's hat, not a sweater. A 1,200-metre cone suggests an adult cardigan. Suggestions reference real free patterns the user can find in Ravelry, Lion Brand's free library, or Drops Design when grounded search is available.
- **The label-found case is honoured, too** — if the user does have a label, photograph it; the app reads the brand, fibre composition, dye-lot, ball-band yardage, and care symbols, and replaces every estimate with the verifiable fact.
- **Community stashes** — invite a knit-night friend or a cousin in Cusco; her stash merges into the shared inventory, with each ball still attributed to its owner.
- **Bound for printing** — export a printable index-card stack, one card per yarn, that the user can keep in the basket on top of the stash itself.
## 4. Features to build
- Camera capture for an entire basket, bin, or shelf at once (mobile-first), with re-photograph + auto-crop guides for blown-out windows or yellow incandescent light
- Single-ball capture mode for the deliberate-cataloguer who wants one ball per photo with the label held alongside
- Upload from photo library or a folder dragged from the desktop (some users have already photographed each ball individually before discovering the app)
- Automatic ball-vs-skein-vs-cake-vs-cone detection — the model distinguishes the wound shapes and adjusts the yardage formula
- Multimodal parse — fibre estimation + colour extraction + weight-class call + yardage estimate + label OCR (when a label is visible) in a single Gemini 3.5 Flash call
- Explicit-confidence surfacing — every estimate carries a `confidence` 0-1 and a one-line `reason`, never a bare claim
- Burn-test / float-test / wet-test prompts — when the model is below 0.7 confidence on fibre, it offers the specific physical test the user can do at the sink or with a candle, and lets her enter the result to update the record
- Label OCR — reads ball-bands in any language the model can read; extracts brand, range, fibre composition, ball-weight in grams, ball-yardage in metres + yards, dye-lot, recommended needle size, recommended gauge, washing instructions, country of origin
- Care-symbol identification — the laundry-care pictograms (washing, bleach, drying, ironing, dry-cleaning) are extracted from the ball-band photograph into structured fields
- Colour-swatch extraction — average colour, dominant colour, and the secondary colours present in heathered or variegated yarns; the hex codes are shown alongside, and the app warns when a photo's white balance looks off
- Sensory metadata — at the user's option, each yarn gets tagged "soft / scratchy", "fluffy halo / smooth", "splitty / cohesive" based on appearance, with the explicit caveat that this is an estimate from the photograph and not a substitute for touching the ball
- Yardage maths shown to the user — the volume estimate from the photograph, the assumed grams-per-metre for the inferred fibre and weight, the resulting metres estimate, all in one tap-to-expand panel
- Project-suitability suggestions — calibrated to yardage remaining and weight class; show real-pattern examples for each suggestion ("with 180m of aran-weight you can knit a 'Sockhead Slouch'-style hat — verify on Ravelry")
- Suggested-substitutes table — for any yarn where the user wants to swap, the app proposes commercial yarns of the same weight and fibre that would substitute well
- Stash-down view — the user marks a yarn "in use on project X" and the inventory grays it out; finished projects free the yarn back into the available pool, minus the metres knit
- Estate / inheritance mode — special intake flow for the basket-from-Grandma case: zero label assumption, every fibre guess marked low-confidence by default, easy bulk-edit when the user remembers a provenance
- Print and export — a printable index-card stack (one card per yarn) for the basket itself; a CSV export for the spreadsheet-keepers; a Ravelry-compatible stash export for users with Ravelry accounts
- Community stash sharing — invite by magic-link email; each ball retains its owner; views can be filtered by "mine", "ours", "available to borrow"
## 4b. Required Gemini capabilities + backend services
**This template's intelligence comes from the Gemini capabilities below. Wire them up explicitly — don't substitute generic LLM calls.**
### Gemini capabilities (the load-bearing intelligence)
- **Multimodal image input** (Gemini 3.5 Flash) — reads a wide-shot of a basket and produces a structured inventory of every visible ball: bounding box per ball, fibre estimate with confidence, colour extraction, weight-class estimate, ball-shape detection, visible volume estimate. The same call also reads a ball-band when one is visible, and parses it into a verified-fact block that overrides every estimate.
- **Structured output / JSON Schema** — the response matches the `StashCapture` schema below. Every estimate carries a confidence band and a reason. The schema is included verbatim in the system instruction and as `responseSchema`. **Convert the Zod schema to Gemini's `Schema` type via the SDK helper** before passing to `responseSchema` — do NOT pass raw Zod. **Numeric `min`/`max` constraints are documentation only inside `responseSchema`; clamp server-side.**
- **Multilingual label OCR** (built into Gemini 3.5 Flash) — ball-bands appear in English, French, German, Italian, Spanish, Portuguese, Japanese, Korean, Mandarin, Tagalog, Quechua-influenced Peruvian Spanish, Mongolian, Welsh, Norwegian, Swedish, Finnish, Polish, Czech, Greek, Turkish, Russian, Arabic, Hebrew, Hindi, Bengali, Urdu, Punjabi, Tamil, Amharic, Swahili, Farsi, Khmer, Vietnamese. The model reads them all and emits the same `LabelFacts` shape regardless of source language, with the original-language text preserved in a `text_verbatim` field.
- **Long context (1M tokens)** — once the stash grows past a hundred balls, the suggested-projects-across-the-whole-stash call sees every ball at once: "with this aran-weight wool here, and the merino sock-weight there, plus the silk-mohair lace, you could knit a Crazy Zauberball cardigan if you re-skein two of the singles". **Guardrail**: a parsed StashEntry averages ~600 tokens; a 250-ball stash ≈ 150k tokens (comfortable). For stashes above 1,200 balls, chunk by weight class or by fibre family before the whole-stash project-suggestion call — the 1M ceiling is real and a 2,500-ball stash will exceed it.
- **Search grounding** — for the project-suggestion call when grounded recommendations are wanted. "180 metres of aran-weight wool, suggest one free pattern" runs against grounded search results from Ravelry's free-pattern index, Drops Design, Berroco, Lion Brand, and Garnstudio. Grounded search prevents the model from inventing pattern names that do not exist on those sites.
- **Gemini TTS** (`gemini-3.1-flash-tts-preview`) — reads the entry aloud for the maker who is winding yarn with both hands and cannot look at a screen: "ball forty-three, aran-weight wool, approximately one hundred and eighty metres remaining, good for a hat". Slow, calm, kitchen-table pace. Pre-process pauses with `…` and paragraph breaks with `—`; do NOT attempt SSML `` or `` — Gemini 2.5 TTS does not support them. Voice locale via `languageCode`; pronunciation follows the voice's native locale.
- **Nano Banana 2 (`gemini-3.1-flash-image`)** — generates a single fallback illustration when a user has not yet photographed her stash and wants a sample card to populate the demo (a warm, hand-drawn-feeling watercolour of a single ball of wool on a wooden table, slight imperfection, not glossy). Used for empty-state and onboarding only — never to fabricate stash content.
- **Thinking levels** — `medium` for the primary basket-parse call (visual fibre estimation + multi-ball bounding + colour extraction + yardage estimation). `low` for label-only OCR, colour extraction on a single ball, and project suggestions. Surface `thoughtSummary` only when the user taps the small "(i) how the app made this guess" icon next to a low-confidence field.
### Backend services
- **Auth — Required.** Firebase Auth with Google sign-in (auto-provisioned by AI Studio Build). **Apple sign-in is optional but user-configured**: it requires an Apple Developer account, Service ID, Key ID, and private key wired into the Firebase Auth console. **Magic-link email** (used to invite a knit-night friend or a cousin into a shared stash) also requires the sender domain to be authorised in Firebase Auth. Stashes are private to the owner and explicitly-invited members. No public-by-default.
- **Database — Required.** Firestore for `users`, `stashes`, `entries`, `labels`, `projects`, `stash_members`, `burn_test_results`.
- **File storage — Required.** Firebase Storage for original yarn photographs (preserved at upload resolution, forever) + cropped per-ball thumbnails. **Storage is NOT auto-provisioned by AI Studio Build today** — enable it in the Firebase console and wire the bucket name into the AIS Build project before first photograph upload. Pre-signed URLs only; the photographs are never publicly addressable.
- **Email — Required (transactional).** Stash-sharing invitations via email link (Firebase Auth magic links). Optional weekly digest of "what's in your stash" for the user who wants the gentle nudge.
- **Payments — Not needed for v1.** Free for personal use. A future "printed stash card pack" tier could pipe to a print-on-demand partner (Moo, Vistaprint) and charge for that physical artefact only.
- **External APIs:** Gemini API for all intelligence; optional Ravelry API for users who want to two-way-sync their stash with their Ravelry account (Ravelry-account OAuth required, user-configured).
**Environment variables:** every secret (Gemini API key, Firebase service-account JSON, optional Ravelry OAuth credentials, optional Stripe key if a printed-card tier is added) lives in environment variables — never in client bundle. Include a `.env.example`.
**Auth + data privacy reminders:** never log secrets · never store passwords in plain text · use HTTPS everywhere · honour 'delete my account' inside the UI · explicit opt-in for any analytics · the user's stash photographs are never sent to Gemini for model training (use the Gemini API on the paid tier, where Google does not use your content for model training, per the Gemini API Additional Terms) · fibre guesses below 0.7 confidence are surfaced as guesses, never as facts.
**Read this first — prompt-craft rules that apply to every call in this template:**
1. **Name the model variant explicitly** in every Gemini API call. Do not let the agent pick the model. See the per-call matrix below.
2. **Pin `thinkingLevel` explicitly** per call. See the matrix.
3. **Seed the JSON Schema as a fenced TypeScript / Zod block** in the system instruction or `responseSchema` field. The literal schema is below. **Convert the Zod schema to Gemini's `Schema` type via the SDK helper** before passing to `responseSchema`. **Numeric `min`/`max` are documentation only inside `responseSchema`; clamp server-side.**
4. **Pin the system instruction separately** from user input. Use the `systemInstruction` field for persona + behavioural rules; use `contents` for user input. Never concatenate.
5. **Pre-declare tools as an enable/disable list** per call. The matrix names which tools are enabled per call. Tools NOT listed for a call should be disabled.
6. **State negative constraints explicitly** — they are listed below. They are NOT "be careful" suggestions; they are hard rules the model must follow.
7. **Grounded responses can wrap JSON in ```json fences or add prose preamble.** Server-side, strip fences and brace-extract:
```typescript
function safeExtractJSON(raw: string): T {
const clean = raw.replace(/```json\s*|```/gi, '').trim();
const s = clean.indexOf('{'); const e = clean.lastIndexOf('}');
if (s === -1 || e === -1) throw new Error('No JSON boundaries in grounded response');
return JSON.parse(clean.slice(s, e + 1)) as T;
}
```
8. **Strip unsupported Zod modifiers before passing to `responseSchema`** — Gemini's OpenAPI subset rejects `.regex()` / `pattern`, fixed-length `z.tuple()`, and other custom validators. Use a sanitizer that flattens tuples to length-2 arrays and removes regex patterns before serializing. Validate those constraints in middleware AFTER parsing.
### Per-call model + tools matrix
| Call | Model | thinkingLevel | Tools enabled |
|------|-------|---------------|---------------|
| Parse basket photo → bounded balls + per-ball estimates → `StashCapture` | `gemini-3.5-flash` | medium | (none) |
| Parse single-ball photo with label visible → `LabelFacts` overrides estimates | `gemini-3.5-flash` | low | (none) |
| Colour extraction on a cropped single-ball thumbnail | `gemini-3.5-flash` | low | (none) |
| Project suggestions for one yarn (yardage + weight + fibre) | `gemini-3.5-flash` | low | `google_search` grounding (no `responseSchema` on this call — see note) |
| Whole-stash project suggestions (long-context across the inventory) | `gemini-3.5-flash` | medium | (none) — long-context over the whole stash |
| Generate TTS narration of a stash entry | `gemini-3.1-flash-tts-preview` | n/a | n/a |
| Generate sample empty-state illustration (one ball of wool) | `gemini-3.1-flash-image` | n/a | n/a |
*Note for builders:* on TTS and image-generation calls, omit `thinkingConfig` entirely — the field is not supported on those models. The `n/a` cells in this matrix are documentation only; do not serialise them into the request body.
*Note on grounded calls:* `responseSchema` and `google_search` grounding are **mutually exclusive** in one Gemini call today. On the grounded project-suggestion call, instruct the model to emit JSON in the text body and parse it server-side. Read citation URLs from `response.groundingMetadata.groundingChunks[].web.uri` — do NOT ask the model to include URLs in the JSON body; it will hallucinate them.
### Primary structured-output schema (seed this verbatim in the prompt)
```typescript
import { z } from "zod";
const ConfidenceBand = z.object({
value: z.number().min(0).max(1),
reason: z.string(), // one sentence, plain language
});
const ColourSwatch = z.object({
hex_average: z.string(), // "#7B4F2A"
hex_dominant: z.string(), // dominant cluster
hex_secondary: z.array(z.string()), // for heathered / variegated
description: z.string(), // "warm oatmeal with rust flecks"
white_balance_warning: z.boolean(), // true if photo looks colour-shifted
});
const FibreEstimate = z.object({
primary_fibre: z.enum([
"wool", "merino", "alpaca", "llama", "vicuña", "mohair", "cashmere",
"qiviut", "angora", "yak", "camel",
"silk", "tussah_silk",
"cotton", "organic_cotton", "mercerised_cotton",
"linen", "hemp", "bamboo_viscose", "tencel",
"abacá", "ramie", "jute",
"acrylic", "nylon", "polyester", "elastane",
"blend_unspecified",
"unknown",
]),
secondary_fibres: z.array(z.string()), // free-text if blend
confidence: ConfidenceBand,
visible_clues: z.array(z.string()), // "matte surface", "long staple", "fluffy halo"
suggested_physical_test: z.enum([
"burn_test", // distinguishes animal / cellulose / synthetic
"float_test", // cotton sinks, wool floats briefly
"wet_test", // wool felts, cotton does not
"stretch_test", // elastic recovery
"none_needed",
]).nullable(),
});
const WeightClassEstimate = z.object({
weight_class: z.enum([
"thread", // 000, cobweb
"lace", // 0
"light_fingering", // 1, sock
"fingering", // 1
"sport", // 2
"dk", // 3, double-knit
"worsted", // 4
"aran", // 4
"bulky", // 5, chunky
"super_bulky", // 6
"jumbo", // 7
"unknown",
]),
wpi_estimate: z.number().nullable(), // wraps-per-inch, if estimable
confidence: ConfidenceBand,
});
const VolumeEstimate = z.object({
ball_shape: z.enum([
"centre_pull_cake",
"pull_skein",
"twisted_hank",
"open_hank",
"donut_ball",
"cone",
"loose_pile",
"unknown",
]),
approx_diameter_cm: z.number().nullable(),
approx_length_cm: z.number().nullable(),
approx_volume_cm3: z.number().nullable(),
visible_only_caveat: z.string(), // "back of basket not visible"
});
const YardageEstimate = z.object({
metres_remaining_estimate: z.number().nullable(),
yards_remaining_estimate: z.number().nullable(),
grams_estimate: z.number().nullable(),
confidence: ConfidenceBand,
calculation_shown_to_user: z.string(), // "visible vol × density-for-aran-wool"
});
const LabelFacts = z.object({
label_detected: z.boolean(),
text_verbatim: z.string().nullable(), // original-language label text
brand: z.string().nullable(),
range: z.string().nullable(), // "Felted Tweed", "Pima Cotton"
colour_name: z.string().nullable(), // "Watery 152"
dye_lot: z.string().nullable(),
fibre_composition_verbatim: z.string().nullable(), // "100% merino"
ball_weight_grams: z.number().nullable(),
ball_yardage_m: z.number().nullable(),
ball_yardage_yd: z.number().nullable(),
recommended_needle_mm: z.number().nullable(),
recommended_gauge_stitches_per_10cm: z.number().nullable(),
care_symbols: z.array(z.enum([
"machine_wash_cold", "machine_wash_warm", "hand_wash", "dry_clean_only",
"do_not_bleach", "bleach_ok", "tumble_dry_low", "do_not_tumble_dry",
"iron_low", "iron_medium", "do_not_iron",
"do_not_wash", "do_not_dry_clean",
"wool_safe_detergent",
])),
country_of_origin: z.string().nullable(),
source_language: z.string().nullable(), // BCP-47, e.g. "ja-JP"
});
const BoundingBox = z.object({
x: z.number(), // 0-1 normalised
y: z.number(),
w: z.number(),
h: z.number(),
});
const StashEntry = z.object({
entry_id: z.string(),
source_photo_uri: z.string(), // Files API `files/*` resource name
bounding_box_in_source: BoundingBox, // where this ball sits in the basket photo
thumbnail_uri: z.string().nullable(), // cropped per-ball image, generated server-side
artefact_type: z.enum([
"ball", "skein", "cake", "cone", "hank", "loose_pile", "scrap", "unknown",
]),
colour: ColourSwatch,
fibre: FibreEstimate,
weight: WeightClassEstimate,
volume: VolumeEstimate,
yardage: YardageEstimate,
label: LabelFacts,
sensory_estimate: z.object({
softness_guess: z.enum(["soft", "medium", "scratchy", "uncertain"]),
halo_guess: z.enum(["smooth", "light_halo", "fluffy_halo", "uncertain"]),
cohesion_guess: z.enum(["cohesive", "neutral", "splitty", "uncertain"]),
caveat: z.string(), // always present: "estimated from photo only"
}).nullable(),
notes_user_visible: z.string(), // "matte surface and long staple suggest wool or alpaca"
flagged_for_user_review: z.array(z.object({
field_path: z.string(),
reason: z.string(),
})),
});
const StashCapture = z.object({
source_photo_uri: z.string(),
artefact_count_estimated: z.number(),
artefacts_visible: z.array(StashEntry),
basket_context: z.enum([
"open_basket", "bin", "shelf", "loose_pile", "single_ball_intake", "unknown",
]),
lighting_warning: z.enum([
"warm_incandescent_shift", "blown_out_window_light",
"fluorescent_cool_shift", "mixed_light_unreliable", "looks_neutral",
]),
global_caveat: z.string(), // "back balls partially occluded; estimates are visible-volume only"
});
type StashCapture = z.infer;
```
### Common failure modes (and how to avoid them)
- Agent silently downgrades `thinkingLevel` on the the basket parse call to save quota — pin `gemini-3.5-flash` with the matrix-specified `thinkingLevel` explicitly. Flash misses smaller balls in the back of the basket, conflates similar colours into one entry, and over-confidently calls "wool" on what is actually a wool / acrylic blend.
- Confidence numbers all default to 0.95 — without explicit instruction, the model will print confident numbers. The system instruction pins a calibration scale: 0.95 only when a label is visible and read, 0.7-0.85 when colour + texture clearly point to a single fibre family, 0.4-0.6 when the model is genuinely uncertain. Surface anything below 0.7 to the user as a guess.
- Hand-spun singles read as "wool" with high confidence — hand-spun yarn from a sheep the user owns is still wool, but the model has no way to know the breed or staple length. Force the model to say "hand-spun, fibre family probable wool, breed indeterminate from photograph" and flag it for the user.
- Yardage estimated from the visible photograph without acknowledging back-of-basket occlusion — pin in the system instruction: "you estimate visible volume only; back-of-basket balls are partially occluded and the estimate is a lower bound". Surface `visible_only_caveat` on every entry.
- Volume estimate ignores ball-shape — a 10 cm centre-pull cake holds more yarn than a 10 cm donut ball. Force the model to commit to a `ball_shape` before estimating volume.
- Label OCR fails silently on non-Latin scripts — Japanese ball-bands, Korean ball-bands, Chinese ball-bands, Cyrillic Russian ball-bands. Pin in the system instruction: "if you can read the label, transcribe `text_verbatim` in the source script; the brand, range, fibre, and gauge fields are filled in from your reading. If you cannot read the label, say so and leave the fields null."
- Colour extraction off because of warm incandescent light — surface a `lighting_warning` on the `StashCapture`. The hex codes are produced anyway, but the warning tells the user "the photo was lit by a yellow bulb; the hex values are warm-shifted by about 15%".
- Project suggestions reference patterns that do not exist — only the grounded call may suggest a named pattern. The ungrounded call suggests project shape and yardage range only ("180m of aran weight is enough for an adult hat or a pair of fingerless mitts") and never names a pattern.
- Heathered or variegated yarns reduced to one hex — preserve `hex_secondary[]` and describe the heathering in `colour.description`.
- Care symbols guessed when not legibly visible — care symbols are extracted only from a ball-band photograph where the pictograms are sharp. If they are not sharp, leave the array empty.
- Estimates rendered to the user as facts — every estimate must render with its confidence band and its reason. The UI never strips the confidence away.
### Negative constraints (hard rules)
- Do NOT label any fibre with confidence ≥ 0.9 unless a legible ball-band has been read. Vision-only estimates cap at 0.85.
- Do NOT name commercial yarn brands ("this is Rowan Felted Tweed") unless a label is visible and the brand is OCR-extracted. The vision-only call never asserts a brand.
- Do NOT suggest a named pattern unless the grounded call returned it with a citation. The ungrounded call describes project shapes only.
- Do NOT silently assume back-of-basket balls match front-of-basket fibres. Each visible ball gets its own entry; occluded balls are not invented.
- Do NOT extrapolate to "good for a sweater" from a 200m ball. Yardage maths is honest; a 200m worsted-weight is a hat, not a sweater.
- Do NOT collapse a heathered or variegated yarn into a single average colour. Surface the colour mix in `hex_secondary[]` and in the description.
- Do NOT use the user's stash photographs to train or fine-tune any model. Use the Gemini API on the paid tier, where Google does not use your content for model training, per the Gemini API Additional Terms.
- Do NOT auto-publish or auto-share. Stashes are private by default. Sharing is explicit, per-stash, per-invitee.
- Do NOT recommend a fibre swap that would change the project's properties without saying so. A wool-for-acrylic swap loses the wool's stretch and bloom; surface that in plain language.
- Do NOT assert sensory properties as facts. "Softness guess" is always a guess from the photograph; the user must touch the ball to know.
### Per-call `systemInstruction` strings
Use these as the literal `systemInstruction` field for each Gemini API call the built app makes. They complement the series-wide rules already uploaded as the global instructions file (`00-series-instructions.txt`).
### Call: Parse basket photo → bounded balls + per-ball estimates → `StashCapture`
Model: `gemini-3.5-flash` · thinkingLevel: medium · Tools: (none)
```
You are a yarn-inventory assistant. The user uploads one photograph
of a basket, a bin, a shelf, or a loose pile of yarn. Your task is
to identify every visible ball, skein, cake, cone, or hank — return
a structured StashCapture matching the schema provided.
You are a careful estimator. You hold every guess lightly. You name
your reasons. You refuse to bluff.
For each visible artefact:
1. Locate it in the photo with a normalised bounding box (x, y, w, h
in 0-1 image-space coordinates).
2. Identify the artefact_type — ball, skein, cake (centre-pull),
cone, hank (twisted or open), loose pile, scrap. The shape
matters: a centre-pull cake holds more yarn at a given diameter
than a donut ball; a cone holds even more.
3. Estimate the fibre composition. The primary_fibre comes from the
enum. Provide a confidence band with a one-sentence reason. Use
the visible_clues array to list what you saw (matte surface, long
staple visible, fluffy halo, smooth lustre, plied vs single).
4. Estimate the weight class — thread, lace, light_fingering,
fingering, sport, dk, worsted, aran, bulky, super_bulky, jumbo.
Provide a wpi (wraps-per-inch) estimate where possible. Provide
a confidence band with a reason.
5. Extract colour: hex_average from the dominant pixels of the
artefact's bounding box, hex_dominant from the largest colour
cluster, hex_secondary as an array of any additional clusters
visible in a heathered or variegated yarn. Describe the colour
in plain language ("warm oatmeal with rust flecks"). If the
photo lighting looks warm-incandescent, fluorescent-cool, or
blown out, set white_balance_warning true.
6. Estimate the volume — diameter and length in cm if estimable,
approx_volume_cm3, and a visible_only_caveat noting any
occlusion (back of basket not visible; ball is partly tucked
into the basket weave).
7. Estimate yardage — metres_remaining and yards_remaining from
the volume × density for the inferred fibre and weight. Show
the calculation in calculation_shown_to_user as one short
sentence. Provide a confidence band.
8. If you can see a ball-band, parse it into LabelFacts. If the
label is in Japanese, Korean, Mandarin, Cantonese, Tagalog,
Spanish, Quechua, Mongolian, Welsh, Norwegian, Polish, Czech,
Greek, Turkish, Russian, Arabic, Hebrew, Hindi, Bengali, Urdu,
Punjabi, Tamil, Amharic, Swahili, Farsi, Khmer, Vietnamese, or
any other script you can read, transcribe `text_verbatim` in
the source script and fill in the parsed fields from your
reading. Note `source_language` in BCP-47. If you cannot
confidently read the label, set `label_detected` true but
leave the parsed fields null and put a note in
flagged_for_user_review.
9. If `label_detected` is true and the fibre composition was
readable, override the vision-only fibre estimate with the
verified facts and raise confidence accordingly. If the label
is unreadable, the estimates stand.
10. Provide a sensory_estimate object with explicit caveats — this
is always a guess from the photograph, and the user must touch
the ball to confirm. If the photo does not show enough detail
for a guess, set the fields to "uncertain".
Calibration of confidence numbers (apply consistently):
- 0.90-1.00 — a legible label has been OCR-read.
- 0.70-0.89 — colour + texture + ply clearly point to a single
fibre family; multiple visible clues align.
- 0.40-0.69 — one or two clues point to a family but other
interpretations are plausible.
- 0.00-0.39 — genuinely uncertain; suggest a physical test in
`suggested_physical_test`.
Anything below 0.70 must appear in flagged_for_user_review with a
plain-language reason.
Set the StashCapture-level `lighting_warning` from the global
photo. Set `global_caveat` to a one-sentence summary of any
basket-wide occlusion or count uncertainty ("the back row is
partially occluded; the estimated artefact_count_estimated is a
visible-only count").
Hard rules:
- Do NOT name commercial brands ("Rowan", "Drops", "Lion Brand")
unless you have OCR-read a label that names them.
- Do NOT label any fibre with confidence ≥ 0.90 unless a label
was read. Vision-only caps at 0.85.
- Do NOT collapse a heathered yarn into one hex. Use
hex_secondary[].
- Do NOT extrapolate yardage to balls you cannot see.
- Do NOT assert sensory properties. softness, halo, cohesion are
always guesses with a caveat field.
- Do NOT translate the source-language label text. Preserve
text_verbatim in the source script.
Output ONLY the StashCapture JSON matching the provided schema.
No commentary.
```
---
### Call: Parse single-ball photo with label visible → `LabelFacts` overrides estimates
Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: (none)
```
You receive a single-ball photograph where the user has held the
ball-band toward the camera deliberately. Your task is to extract
every fact on the ball-band into the LabelFacts shape.
Read the label carefully. Many ball-bands are folded over the ball
and the visible side may show only some fields; do not invent the
fields you cannot see. Many ball-bands repeat the same information
in multiple languages; pick the user's language if available, else
read whichever language you can read most reliably, and note
source_language in BCP-47.
Extract:
- text_verbatim: every legible character on the visible side of the
label, in the source script, including the brand and range
banner.
- brand: the manufacturer ("Rowan", "Drops", "Hilos LM", "Cascade",
"Manos del Uruguay", "Hilo Sagrado", "Yamatobo").
- range: the product line within the brand ("Felted Tweed", "Pima
Cotton", "Big Wool").
- colour_name: the dye name or number printed on the band.
- dye_lot: the dye-lot code, often a short alphanumeric.
- fibre_composition_verbatim: as printed, "100% merino" or
"50% cotton 50% linen" — preserve order and percentages.
- ball_weight_grams, ball_yardage_m, ball_yardage_yd: from the
printed numbers.
- recommended_needle_mm: from the printed needle pictogram or
"5 mm" text.
- recommended_gauge_stitches_per_10cm: from the printed gauge
swatch ("22 sts × 28 rows / 10 cm").
- care_symbols: from the laundry-care pictograms. Use the closed
enum in the schema. If a pictogram is ambiguous, leave it out
of the array — do not guess.
- country_of_origin: as printed.
Hard rules:
- Preserve text_verbatim in the source script.
- Do NOT translate the printed range or colour name.
- Do NOT invent fields not visible on the label.
- If you can read the fibre composition with high confidence, that
overrides any vision-only fibre estimate.
- If the ball-band is in a language you cannot read confidently,
set label_detected true, transcribe text_verbatim character by
character to the best of your ability, and leave the parsed
fields null with a note that the user should re-photograph or
enter manually.
Output ONLY the LabelFacts JSON.
No commentary.
```
---
### Call: Colour extraction on a cropped single-ball thumbnail
Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: (none)
```
You receive a tight crop of a single ball of yarn. Return a
ColourSwatch JSON object: hex_average, hex_dominant, hex_secondary
(up to four values for variegated or heathered yarns), description
(one short sentence), and white_balance_warning.
Hard rules:
- For solid yarns, hex_average and hex_dominant should be very
close; hex_secondary may be empty.
- For heathered yarns (multiple small colour flecks), surface at
least two hex_secondary values.
- For self-striping or variegated yarns, surface up to four
hex_secondary values capturing the major bands.
- Describe the colour in plain language a knitter would use:
"warm oatmeal with rust flecks", "deep teal with copper
glimmer", "muted plum heathered with grey".
- If the photo's lighting looks warm-incandescent (yellow cast),
fluorescent-cool (green-blue cast), or blown out (clipped
highlights), set white_balance_warning true and note in the
description.
Output ONLY the ColourSwatch JSON.
No commentary.
```
---
### Call: Project suggestions for one yarn (yardage + weight + fibre)
Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: `google_search` grounding
```
You suggest one to three real, free knitting or crochet patterns
that fit a single yarn the user owns: a known weight class, an
estimated fibre, and a known metres_remaining.
Search grounded against Ravelry free-pattern listings, Drops
Design, Berroco free patterns, Lion Brand free patterns, and
Garnstudio. Cite every pattern with a URL — those URLs are taken
from the response's `groundingMetadata.groundingChunks[].web.uri`
field server-side. Do NOT include URLs in the JSON body of your
response; the server will attach them.
Constraints:
- The suggestion must fit the yardage. Do not suggest a sweater
for 200m. Calibrate: hat ≈ 120-220m worsted; mittens ≈ 120-180m
worsted; cowl ≈ 150-300m worsted; baby cardigan ≈ 400-600m dk;
adult cardigan ≈ 900-1500m dk depending on size.
- The suggestion must fit the fibre. Do not suggest a lace shawl
in pure cotton (cotton drapes wrong for lace). Do not suggest
socks in pure silk (silk wears through fast).
- If the yardage falls between common project sizes, say so. "200m
is a generous hat or a small cowl, not a sweater."
Output the response as JSON in the text body (NOT via
`responseSchema` — `responseSchema` and `google_search` cannot be
combined in the same Gemini call today). The JSON shape:
{
"suggestions": [
{
"project_type": "hat" | "mittens" | "cowl" | "shawl" |
"baby_cardigan" | "adult_cardigan" | "scarf" |
"blanket_square" | "socks" | "small_accessory",
"pattern_name_if_grounded": "string or null",
"yardage_needed_m": number,
"yardage_fits": "comfortable" | "just_enough" | "tight",
"fibre_fits": "ideal" | "workable" | "compromise",
"one_line_reason": "string"
}
],
"general_note": "string"
}
Server-side: parse the JSON, then read citation URLs from
`response.groundingMetadata.groundingChunks[].web.uri` and attach
them to each suggestion by matching pattern_name. Do NOT ask the
model to include URLs — it will hallucinate them.
No commentary outside the JSON.
```
---
### Call: Whole-stash project suggestions (long-context across the inventory)
Model: `gemini-3.5-flash` · thinkingLevel: medium · Tools: (none, long context)
```
You receive the entire stash as a list of StashEntry objects. Your
task is to suggest two to five projects that use the stash well —
either using one substantial entry, or combining two or three
complementary entries (a main colour + a contrast + a heel-and-toe
fingering, for example).
Constraints:
- Calibrate yardage to project size (see the per-yarn call's
calibration).
- When combining entries, the weights must match within one class.
An aran-weight cannot be substituted for a fingering.
- When combining entries, prefer fibres that wash similarly. Do
not pair a superwash merino with a non-superwash for stripes
unless you warn the user about uneven shrinkage.
- Order the suggestions by yarn-utilisation — projects that use
the largest yardages first, scrap-busters last.
- Reference the specific entries by entry_id.
Hard rules:
- Do NOT name commercial patterns unless they appear in the stash's
metadata (the user may have noted "intended for the Sockhead
Slouch pattern"). The grounded per-yarn call is where named
patterns come from.
- Do NOT propose a project that would deplete two entries the
user has marked "keepsake" or "not for use".
- Surface every project with a one-line "why this combination"
reason and a one-line "what to verify before casting on"
reason.
Output: an array of project_suggestions[] as a single JSON object,
with entry_ids referenced. No commentary outside the JSON.
```
---
### Call: Generate TTS narration of a stash entry
Model: `gemini-3.1-flash-tts-preview` · n/a · n/a
```
Voice: warm, unhurried. Pick the Gemini 2.5 Flash TTS voice whose
`languageCode` matches the user's interface language — pronunciation
will follow that locale automatically. Pre-process the text input:
- Open with the entry's index in the basket ("Ball forty-three…").
- Then weight class, fibre estimate (with the confidence stated in
plain language: "a confident wool", "a probable wool-alpaca
blend"), and yardage estimate ("approximately one hundred and
eighty metres remaining").
- Close with one suggested project ("…good for a hat or a pair of
fingerless mittens").
- At each clause break insert a single ellipsis (`…`). At paragraph
breaks insert a blank line plus an em-dash (`—`). Gemini 2.5 TTS
does not support SSML `` — these textual cues are how
you signal pace.
- Skip the hex codes, the bounding-box coordinates, the
visible_only_caveat — they are not spoken content.
- Target rate: ~110 words per minute — kitchen-table pace, not
podcast pace.
Style direction: prepend ONE short directive sentence to the
text input, exactly like: "Read warmly and unhurriedly, as a
yarn-shop owner reading out the basket inventory to a friend who
is winding skeins. …". There is no separate `style` API field on
Gemini 2.5 TTS; the directive sentence inside the input is how
style is conveyed.
Phoneme overrides (Welsh ll, Quechua glottal stops, Mongolian khoit
vowels) are NOT exposed by Gemini 2.5 TTS — no SSML ``
tag. Pronunciation comes from the chosen voice's native locale.
Mid-call voice switching is not supported.
```
---
### Call: Generate sample empty-state illustration (one ball of wool)
Model: `gemini-3.1-flash-image` · n/a · n/a
```
Generate one warm, hand-drawn-feeling watercolour illustration of a
single ball of cream-coloured wool resting on a worn wooden surface
in afternoon light. Soft edges, slight imperfection in the linework,
a hint of shadow under the ball. No text, no labels, no UI chrome.
Aspect ratio roughly 4:3. The illustration is used for the empty
state on the welcome screen; it sets a kitchen-table mood, not a
product-photography mood.
Do NOT generate a photo-realistic ball; the empty-state image must
look obviously illustrated so the user is not confused into thinking
it represents real stash content.
No commentary. Image only.
```
## 5. Use cases & content to include
Build dedicated UI sections or flows for each of these — they tell you what content the app must support.
- **The basket-on-the-floor capture.** A user has emptied her under-the-bed bin onto the rug and pointed her phone straight down. The app processes the wide-shot, identifies thirty-two visible artefacts, and queues each for refinement. The first card surfaces immediately ("the centre-pull cake of cream — confident wool, 320m, soft-confidence label visible upper-right edge") and the rest stream in as the model finishes each.
- **The inherited cabinet.** A Filipino-American knitter in San Jose has just brought home her late tita's yarn cabinet. There are eighty balls, none labelled (her tita removed every ball-band because "they got in the way of the colour"). The app routes her into Inheritance Mode: every fibre guess defaults below 0.7 confidence; every card surfaces "this is a guess; tap to refine when you remember"; the global caveat at the top reads "this stash is a memory, not a label".
- **The hand-spun stash.** A Peruvian collector in Cusco is cataloguing three generations of hand-spun alpaca singles and plied yarns, the older ones from her grandmother's flock. The app says "hand-spun, fibre family probable alpaca, breed indeterminate from photograph" and offers a notes field where she can record what she knows that the model cannot ("this one is from Llamita, the older brown jenny").
- **The label-in-hand intake.** A Norwegian-British knitter in Edinburgh holds each new skein with the ball-band visible and photographs them one at a time. The label OCR call resolves the brand, range, fibre, dye-lot, ball-band yardage, and the laundry-care pictograms, then writes a 0.97-confidence entry without ever invoking the vision-only fibre estimator.
- **The flood-damaged box.** A user pulls a sodden box from a basement after a pipe burst. Half the labels are gone; some of the yarn is felted. The app handles the half-felted balls with grace — surface that the surface texture suggests felting, flag the entries for the user, and offer the option to bulk-tag the lot "salvage / pet bedding only" rather than catalogue at high confidence.
- **The Tuesday-night community cabinet.** A librarian who runs the weekly knit-and-natter inherits two hundred donated balls. The app routes her into Stewardship Mode: every entry is logged "available to borrow", a printable bin-tag QR code is generated per ball, and members of the group can search the inventory from their phones.
- **The variegated-yarn case.** An indie-dyed sock-yarn skein in a wild self-striping colour-way. The colour-extraction call returns five secondary hex codes, the description reads "deep teal, copper, navy, mustard, with cream flashes between the bands", and the project-suggestion call says "this is calling for a shawl that lets the colour change tell the story; do not pool".
- **The cone-from-the-mill.** A weaver has a 1.2 kg cone of unmercerised cotton. The model identifies the cone shape, applies the cone yardage formula (much more yarn per visible diameter than a ball), and surfaces "approximately 4,800m, enough for a placemat set or a small bath towel".
- **The estate sale haul.** A user posts twenty-four lot-bought skeins onto a sheet on the kitchen table. The app produces a structured inventory she can use to sell on the unwanted ones — "skein 11: aran-weight, probable acrylic-wool blend, 180m, blue heathered with grey, suitable for charity-knit hats; estimated resale value $4-6".
- **The Ravelry-syncer.** A user with a long-standing Ravelry stash account wants to bring it into the app. The Ravelry OAuth flow imports her existing stash; the app adds the per-ball photographic capture on top and pushes refinements back to Ravelry on demand.
- **The TTS basket-readout.** A spinner is plying singles on her wheel and cannot look at the screen. She taps "read me the basket"; Gemini TTS reads each entry in turn, slow and clear, in her interface language. She stops the readout when she hears the entry she wants and asks Gemini to mark it "in use".
- **The substitute search.** A user has half a pattern done in a discontinued Rowan colourway and needs more. The app's "find me a substitute" flow searches the user's own stash first ("you have 380m of a matching aran-weight in your 'inherited' bin"), then proposes commercial substitutes from grounded search.
## 6. Page structure
Build the following screens / sections in this order. Adjust copy to fit the voice, but keep the structural intent.
1. **Welcome / sign-in.** A warm watercolour-styled illustration of a single ball of cream-coloured wool on a wooden table (generated via Nano Banana 2 per the empty-state prompt above). One paragraph: "Yarn Stash turns a basket of unlabelled yarn into an inventory you can actually use — every ball photographed, every guess held lightly, every original preserved." Single Google sign-in button; Apple sign-in next to it. Below: "Try with the sample stash" → loads the demo stash in section 8a.
2. **Empty state — "Start a stash".** Three big input methods: 📷 Wide-shot capture · 🏷 Label-in-hand · 🖼 Upload from library. A short explainer below each ("Best for cataloguing a whole basket at once", "Best for new yarn with the label still on", "Best if you've already photographed each ball").
3. **Wide-shot capture flow** (mobile-first). Live viewfinder with framing guides. The flow encourages a top-down shot of the open basket. After capture, a queue card appears at the bottom: "Reading the basket… 32 balls visible so far." As the model finishes each entry, a thumbnail appears in the queue. The user can refine any entry now or later.
4. **Label-in-hand capture flow.** Single-ball mode. The user holds the ball with the label visible to the camera. The viewfinder auto-detects the band and asks the user to hold steady; the OCR call runs and the entry is written at high confidence directly.
5. **Processing queue.** A vertical list of the session's batch. Each item shows the cropped thumbnail, the dominant colour swatch, an honest step bar: "Estimating fibre…" → "Reading the colour…" → "Measuring the volume…" → "Estimating yardage…". Each step takes 1-3 seconds; the wide-shot batch processes in parallel and finishes in 20-40 seconds for a typical basket.
6. **Yarn detail view.** A three-column layout on desktop, stacked on mobile. Left column: the photograph (zoomable; if a label was read, the label crop is shown beneath the main photograph). Middle column: the estimates — fibre, weight, colour, volume, yardage — each rendered with its confidence band and a "(i) how the app made this guess" affordance that surfaces the model's `visible_clues` and the calculation it showed. Right column: project suggestions (the per-yarn grounded call's output, with pattern links). Sticky header: ball ID → estimated fibre + weight chip → confidence chip → "(i) how the app made this guess".
7. **Stash view.** A magazine-grid of all entries. Filter by weight class, fibre family, colour (a colour-strip filter), confidence band ("show me everything below 0.7"), in-use vs available. Sort: by colour, by weight, by yardage, by date added, by confidence.
8. **Colour view.** All yarns laid out on a 2D colour-space (hue × value). Tapping a dot opens the yarn. Useful for project planners deciding on a palette.
9. **Project planner.** The whole-stash project-suggestion call's output. Each suggestion lists the entries it combines, the yardage maths, the "what to verify before casting on" note, and a "save to my projects" button.
10. **Sharing & community stash.** Modal: "Invite a knit-night friend or a cousin to your stash". Magic-link email; arrival drops the invitee straight into the same stash with her own avatar. Per-ball owner attribution is preserved.
11. **Inheritance Mode.** A separate intake flow for the basket-from-Grandma / inherited-cabinet case. The flow opens with a full-screen note: "This stash is a memory, not a label. Every guess starts low-confidence; tap any card to refine it when you remember." All entries from this flow default to confidence 0.5 and surface a softer card design with extra room for user notes.
12. **Stewardship Mode.** Variant flow for community-cabinet stewards. Every entry is logged "available to borrow"; a printable bin-tag QR code is generated per ball. Members can search the inventory from their own phones via a shared sign-in or guest link.
13. **Printable index-card export.** A side-by-side typeset preview. Choose card density (1 / 2 / 4 per A4 page). Each card shows the thumbnail, the colour swatch, the weight, the fibre estimate with confidence, the yardage estimate, and one suggested project. Toggle: include hex codes, include label OCR text. Export PDF; future tier: print-bound card pack.
14. **Settings & account.** Delete the whole stash forever (60-second confirmation). Export everything (CSV + photographs zipped + JSON). Revoke a cousin's access. Capabilities `(i)` icon in header.
15. **Footer.** "Made for the basket nobody else could read." Privacy: "Your stash is yours. We never train on it." Capabilities `(i)` icon in header.
## 6b. First-visit onboarding
Show a **first-visit onboarding** the first time a visitor lands on the app (detect via `localStorage` flag; do not show on return visits). Three slides, dismissible at any time. Persistent re-entry: a `?` icon in the header reopens it.
**Slide 1 — What this is.**
- Headline: "Welcome to Yarn Stash."
- Subhead: "Turn a basket of unlabelled yarn into a working inventory — in any fibre, any weight, any colour, with every guess held lightly and every original photograph preserved."
- One paragraph (≤ 60 words) explaining who this is for: knitters, crocheters, spinners, weavers, felters whose stash has outgrown their memory; and what makes it different from a generic photo-tagger — it gives you a confidence number on every guess and offers the burn test or float test when it isn't sure.
- Visual: a small annotated illustration of a single ball of wool with the relevant data fields labelled (fibre estimate, weight class, colour swatch, yardage estimate, label OCR) — not a generic photo icon.
**Slide 2 — Try it now.**
- One short prompt: "Try with the sample stash".
- A live demo input pre-loaded with the four sample entries from section 8a (the inherited Filipino-American basket, the Peruvian hand-spun alpaca, the Edinburgh new-purchase with label, and the community-cabinet donation lot).
- 1-2 sentences pointing at *the specific page elements* where the Gemini magic happens (the confidence chip on the inherited basket's cards; the label-OCR overlay on the new-purchase; the multi-entry bounding-box overlay on the basket wide-shot).
**Slide 3 — How to remix this.**
- Headline: "Make this yours."
- Three short bullets:
- "Swap the sample stash in `/data/seed-stash/` for your own photographs."
- "Adjust the prompts in `/server/prompts/` to fit the fibres and brands you actually have."
- "Wire up your Gemini API key and Firebase project via the env-var list in the capabilities panel."
- Primary CTA: "Use this template" → links to AI Studio Build remix entry point.
- Secondary: "Just exploring — close" (sets localStorage flag, never auto-shows again).
**Accessibility:** focus trap, `Esc` closes, `role="dialog"`, `aria-modal="true"`, `aria-labelledby`, focus restored to trigger on close. Respect `prefers-reduced-motion`.
**Don't:**
- Don't gate content behind the modal. The page beneath must be fully usable.
- Don't auto-reshow on return visits. Use `localStorage['onboarding-seen-v1']`.
- Don't include unrelated CTAs (newsletter signup, social follow). Keep it about the template only.
## 6c. Capabilities info button (persistent in header)
Add a persistent `(i)` icon in the top-right of the header (next to the primary nav). Click → opens a modal/panel titled **"What powers this app"**.
**Panel contents (in this order):**
**Gemini capabilities used (the hero list):**
- **Gemini 3.5 Flash (multimodal)** — reads a wide-shot of a basket and produces a per-ball inventory: bounding box, fibre estimate with confidence, weight class, colour extraction, yardage estimate. The same call also reads ball-bands in any language it can read.
- **Gemini 3.5 Flash (multilingual label OCR)** — ball-bands in English, French, German, Spanish, Italian, Japanese, Korean, Mandarin, Tagalog, Quechua-influenced Peruvian Spanish, Mongolian, Welsh, Norwegian, Polish, Czech, Greek, Turkish, Russian, Arabic, Hebrew, Hindi, Bengali, Urdu, Punjabi, Tamil, Amharic, Swahili, Farsi, Khmer, Vietnamese.
- **Gemini 3.5 Flash (long context)** — once your stash grows, the whole-stash project planner sees every ball at once.
- **Gemini 3.5 Flash + grounded search** — proposes real free patterns (Ravelry, Drops, Lion Brand, Berroco, Garnstudio) with citation URLs that the server attaches from grounding metadata.
- **Gemini 3.5 Flash (colour)** — extracts colour swatches on single-ball thumbnails, including heathered and variegated breakdowns.
- **Gemini TTS** — reads the basket aloud at kitchen-table pace, so you can catalogue while your hands are busy.
- **Nano Banana 2 (`gemini-3.1-flash-image`)** — generates the empty-state illustration only. Never used to fabricate stash content.
- **Firebase Auth** — Google and Apple sign-in, magic-link email for stash sharing.
- **Firestore** — stores your stash, syncs across devices in real time.
- **Firebase Storage** — keeps the original photographs at upload resolution, forever.
- **Cost note** — see the detailed breakdown in 6d. A typical basket capture (40 balls in one wide-shot) costs about $0.05 of Gemini API spend.
- **Privacy note** — your stash is private to you and the friends you invite. This app uses the Gemini API on the paid tier, where Google does not use your content for model training, per the Gemini API Additional Terms. Every fibre guess is presented with a confidence number; the app never tells you it knows something it doesn't.
**Backend services this app depends on:**
- Auth: see section 4b. **Apple sign-in** requires an Apple Developer account + Service ID config in the Firebase Auth console (user-configured). **Magic-link email** requires the sender domain to be authorised in Firebase Auth (user-configured).
- Database: see section 4b
- Storage: see section 4b. **Firebase Storage is NOT auto-provisioned by AI Studio Build today** — enable it in the Firebase console and wire the bucket name into the project before first photograph upload.
- Email: see section 4b
- Payments: see section 4b (not used in v1)
- External APIs: see section 4b (optional Ravelry OAuth)
**Environment variables you'll need to configure:**
- `GEMINI_API_KEY` — your Google AI Studio API key
- `FIREBASE_PROJECT_ID` — your Firebase project id
- `FIREBASE_SERVICE_ACCOUNT` — service-account JSON (server-side only)
- `RAVELRY_OAUTH_CLIENT_ID` / `RAVELRY_OAUTH_SECRET` — optional, only if you want two-way sync with Ravelry stashes
**Cost + privacy notes:**
- A wide-shot capture of 40 balls processed at `gemini-3.5-flash` medium-thinking costs about $0.05. A label-in-hand capture costs about $0.002 per ball.
- The whole-stash project planner runs on demand (default: weekly) and costs about $0.20 per run on a 250-ball stash.
- Where the data lives: your Firebase project, in your Google Cloud account. How to delete it: Settings → "Delete this stash forever" — gone in 60 seconds. What is never sent for training: nothing. The Gemini API paid tier does not train on user content (per the Gemini API Additional Terms).
**Documentation links:**
- AI Studio Build docs
- Gemini API multimodal, multilingual, long-context, TTS, image-generation docs
- Firebase Auth, Firestore, Firebase Storage docs
- A short note on Ravelry OAuth for two-way sync
**Accessibility:** same standards as the onboarding modal — focus trap, `Esc`, ARIA, restored focus.
**Behaviour:**
- Always available — single click from anywhere in the app.
- Tooltip on the `(i)` icon: "How this app is built".
- Mobile: opens as a full-screen sheet that slides up.
- Should be the most honest part of the app — never hand-wave service requirements; never say "AI" without naming the specific Gemini model and capability.
## 6d. Detailed cost breakdown (deployer reads this BEFORE shipping)
- **Wide-shot basket parse (Gemini 3.5 Flash, medium thinking)** — typical 40-ball wide-shot, ~1 image input, ~3,500 output tokens. ~$0.05/capture.
- **Single-ball label parse (Gemini 3.5 Flash, low thinking)** — ~1 image, ~400 output tokens. ~$0.002/ball.
- **Colour-extraction call (Gemini 3.5 Flash, low thinking)** — ~1 image, ~150 output tokens. ~$0.0003/ball. Cached per thumbnail.
- **Per-yarn project suggestion (Gemini 3.5 Flash + grounded search)** — ~$0.001/yarn. Cached per yarn until the yarn's metadata changes.
- **Whole-stash project planner (Gemini 3.5 Flash, medium thinking, long-context)** — runs weekly by default. ~$0.20 per run on a 250-ball stash.
- **TTS narration (Gemini 2.5 Flash TTS)** — billed per output token (~$10/M output tokens), effectively ~$0.000003/character. A 60-word entry narration ≈ $0.001 per entry. Cached per entry.
- **Empty-state illustration (Nano Banana 2)** — ~$0.03/image. Generated once, ships in the public bundle thereafter.
- **Expected cost on a first-time basket capture:** ~$0.05 for 40 balls. **Ongoing weekly whole-stash refresh** (250 balls): ~$0.20/week.
- **Image storage:** Firebase Storage standard tier, ~$0.026/GB/month. A typical phone-camera wide-shot is ~4 MB; a 250-ball stash with both wide-shots and per-ball thumbnails uses ~1.5 GB ≈ ~$0.04/month.
## 7. Design language
- **Mood:** A working stash basket on the floor, the maker sitting on the rug with her knees up, sorting by feel. Warm afternoon light through a window. Not a tech product. Not a yarn-shop catalogue. The basket itself is the hero; the screen is a quiet assistant.
- **Typography:** Display serif for entry titles and the basket overview headings (Source Serif Pro or Adobe Caslon Pro). A hand-drawn accent (sparingly) only for confidence-band labels ("a confident wool") and the user's own added notes — never for the model-generated estimate text, which sits in serif. Clean grotesque for app chrome (Inter or Geist).
- **Palette:** Undyed-wool background `#F3EEE3` for the main canvas, deep-walnut `#2A1F19` for body text, indigo accent `#3D5A8C` for primary actions and confidence chips, madder-red `#A33A2C` only for the warnings (lighting issues, low-confidence fibre flags). A muted ochre `#B98A3A` for the user's own annotations so they read distinct from the model's estimates. Borrowed from naturally-dyed wool, not from SaaS design systems.
- **Imagery:** The photographs of the yarn are the hero. Never crop them tighter than the user did. Per-ball thumbnails respect the natural shape of the ball; ball cards have rounded corners that echo the curve of yarn itself. Colour swatches sit beside the photograph, never replacing it. Empty states use the Nano-Banana watercolour ball; populated states use the user's own photographs.
- **Hand-feel touches:** A barely-visible woven-paper texture on the main background. The "(i) how the app made this guess" expandable panel slides the model's reasoning in with a thin shadow — like lifting a tag off a ball. Hover on a confidence chip reveals the calibration scale (0.95+ = label read; 0.7-0.85 = visual estimate; below 0.7 = guess). Never aggressively glow.
- **Spacing:** consistent 4-px base. Generous whitespace — the yarns need room to breathe so the colours don't bleed visually.
- **Radius:** consistent token set (e.g. 8 / 14 / 22 px). Yarn cards use 14; basket-overview tiles use 8; welcome card uses 22.
- **Shadows:** subtle, layered, warm-tinted. Avoid heavy drop-shadows.
- **Motion:** purposeful — entrance fades, hover lifts, page transitions. Respect `prefers-reduced-motion`. No bouncing animations. No theatrical hero animations. The Colour view's hue-shuffle when a new entry is added is the one place where motion carries meaning; respect reduced-motion by jumping rather than animating.
- **States:** every interactive element has hover, focus, active, disabled. Loading uses skeletons that match the eventual yarn-card layout, not spinners. Empty states have helpful next-action guidance ("Photograph the basket to start", "Hold one new skein up with the label facing the camera").
## 8. Content generation rules
- Write **realistic, specific copy**. NO Lorem Ipsum. NO generic placeholders like 'Your tagline here'.
- Invent plausible names, photographs, fibre estimates, confidence bands, and project suggestions that fit the domain (use the seed content in section 8a as a starting point). When inventing, lean on real fibre-and-yarn vocabulary — Aran, fingering, DK, worsted, alpaca, qiviut, mercerised cotton, abacá — but never claim that a fictional yarn is a real commercial product unless the grounded call returned it.
- Tone: warm, direct, free of corporate language. This template is for a maker, not a yarn brand.
- Headlines: punchy and concrete. No 'Empower your creativity' filler. No 'Revolutionize'. No 'Seamless'.
- Body copy: short paragraphs (2-4 sentences). Use lists where appropriate.
- Plain language. Avoid jargon — except where the user already speaks the jargon (the spinner wants to see "wpi", "staple length", "ply structure"; the knitter wants to see "fingering", "swift", "dye-lot").
- Where the app outputs AI-generated content, never label it as "AI says" — let it speak naturally, with the confidence band as the honesty signal. Low-confidence estimates show as a faintly-underlined fibre name; tapping reveals the alternates the model considered.
## 8a. Seed content (use these specific examples)
Anchor every generated copy + sample data point in the concrete content below. Use these names, numbers, and snippets verbatim where helpful, or generate close variants that sit in the same world.
**Sample stashes (sidebar):**
- "Tita Lorna's Cabinet" (62 entries, contributor: me) — inherited Filipino-American stash from a tita who removed every label "because they got in the way of the colour". Mixed weights, mixed fibres, mostly wool and acrylic blends with a few mercerised cottons. San Jose, 2026.
- "Cusco Alpaca Three Generations" (38 entries, contributors: me, my mother, my abuela) — hand-spun alpaca singles and plied yarns, three generations of women in one family, the older ones from her abuela's flock outside Cusco.
- "Edinburgh New Stash" (24 entries, just me) — recently-purchased skeins with every ball-band still on. Used as the label-OCR demo stash because every entry confidently overrides the visual estimator.
- "Tuesday-Night Cabinet" (180 entries, contributors: 11 knit-night members) — community library stash in stewardship mode, with bin-tag QR codes and per-ball owner attribution.
**Sample yarn entry in detail view (this is what the demo should show):**
- **Source photo:** wide-shot of the inherited cabinet's third shelf, top-down, afternoon window light
- **Bounding box in source:** x: 0.42, y: 0.31, w: 0.18, h: 0.21 (centre-back of the photograph)
- **Artefact type:** centre_pull_cake
- **Colour:**
- hex_average: `#A6845C`
- hex_dominant: `#A6845C`
- hex_secondary: `["#7B4F2A", "#C8AB85"]`
- description: "warm oatmeal with rust flecks"
- white_balance_warning: false
- **Fibre estimate:**
- primary_fibre: "wool"
- secondary_fibres: ["possibly alpaca"]
- confidence: 0.72 — "matte surface and long staple suggest a wool or wool-alpaca blend, but no label visible"
- visible_clues: ["matte surface", "long visible staple", "plied two-ply structure", "slight halo"]
- suggested_physical_test: "burn_test"
- **Weight class:**
- weight_class: "aran"
- wpi_estimate: 8
- confidence: 0.82 — "ply diameter and lay against the basket weave consistent with aran"
- **Volume:**
- ball_shape: "centre_pull_cake"
- approx_diameter_cm: 11
- approx_length_cm: 8
- approx_volume_cm3: 760
- visible_only_caveat: "front of cake visible; back not measurable"
- **Yardage:**
- metres_remaining_estimate: 180
- yards_remaining_estimate: 197
- grams_estimate: 120
- confidence: 0.68 — "volume estimate is visible-only; back of cake hidden by basket weave"
- calculation_shown_to_user: "760 cm³ × 0.16 g/cm³ (aran-wool density) ≈ 120 g; 120 g × 1.5 m/g ≈ 180 m"
- **Label:** label_detected: false, all fields null
- **Sensory estimate:**
- softness_guess: "medium"
- halo_guess: "light_halo"
- cohesion_guess: "cohesive"
- caveat: "estimated from photograph only — touch the ball to confirm"
- **Notes user-visible:** "Warm oatmeal aran-weight wool with a slight halo. No label, so the fibre is a confident guess but not a fact — a burn test would confirm. Probable hat or fingerless-mittens yardage."
- **Flagged for user review:**
- field_path: "fibre.primary_fibre", reason: "no label visible; confidence below the 0.85 vision-only cap"
- field_path: "yardage.metres_remaining_estimate", reason: "back of cake occluded; estimate is a lower bound"
**Sample input artefacts (for the build to demonstrate):**
- A top-down wide-shot of an open basket on a rug, 32 balls visible, mixed colours, afternoon window light. (The Tita Lorna's Cabinet hero shot.)
- A single-ball intake photograph of a new merino skein with the ball-band held in the same hand as the ball. (The Edinburgh stash intake.)
- A close crop of a hand-spun alpaca single with no label visible. (The Cusco stash entry.)
- A wide-shot of a community-cabinet shelf with 28 balls and a printed bin tag stuck to the side. (The Tuesday-Night Cabinet stewardship example.)
- A close crop of a heathered variegated sock-yarn skein, deep teal with copper and navy. (The variegated-yarn demo.)
**Sample voice copy:**
- Onboarding: "Photograph your basket. We'll guess at every ball — and tell you exactly how confident we are."
- Processing: "Reading the basket…" / "Estimating fibres…" / "Measuring colours…" / "Doing the yardage math…"
- Empty stash: "This stash is waiting for its first photograph. Point your phone at the basket to start."
- Error (couldn't make sense of the photo): "We couldn't pick out individual balls in this photo — the lighting may be too dim or the basket too dense. Want to try a clearer wide-shot, or photograph one ball at a time?"
- Save confirmation: "Added to Tita Lorna's Cabinet — warm oatmeal aran-weight wool, 180m estimated."
- Low confidence note: "We're not sure about the fibre here. Tap to see why, or do the burn test at the sink."
- Lighting warning: "This photo looks lit by a yellow bulb. The colours in the inventory will be slightly warm-shifted. Re-photograph in daylight for true colour."
- Label OCR success: "Read the label cleanly — this entry's fibre is a fact, not a guess."
**Sample stash-share invitation email subject + body:**
- Subject: "Marisol — I'm cataloguing Tita Lorna's yarn. Want to add yours from Cebu?"
- Body: "Hi Marisol — I finally pulled out Tita Lorna's cabinet and I'm photographing every ball. Could you add the ones your mom has on her shelves? Tap to join — your balls stay attributed to you, mine stay attributed to me." [Open Stash]
## 9. Media & assets
- **Hero image (landing screen):** A warm watercolour-styled illustration of a single ball of cream-coloured wool on a worn wooden surface in afternoon light. Generated via Nano Banana 2 per the empty-state prompt in section 4b. The illustration looks obviously illustrated — not a fake photograph.
- **App icon / wordmark:** Set in the display serif. A small woven-paper texture behind it. No icon — just type.
- **Empty-state illustration:** Same Nano Banana 2 watercolour ball. Used on the welcome screen and any empty stash view.
- **Demo basket photographs:** Generated per the seed content prompts in section 8a — Nano Banana 2 prompts that specifically request "top-down phone-camera wide-shot of a basket of yarn on a rug, warm afternoon window light, no people in frame, slight imperfection in the photograph". Each demo photograph should look phone-camera-natural, not glossy.
- **Per-ball thumbnail crops:** Generated server-side from the wide-shot bounding boxes, then stored in Firebase Storage. Square 1:1 aspect ratio with a small padding margin.
- **Pattern preview images:** When the grounded project-suggestion call returns a pattern name with a URL, fetch the pattern's preview image from that URL server-side (with attribution) and cache it.
- **Stock fallbacks:** If image generation fails on the empty-state ball, fall back to the photographed sample ball from `/public/samples/sample-ball.jpg`. Never to a "🧶" emoji.
- **Generated imagery:** prefer Nano Banana 2 over stock photography for empty-state and demo. Prompt for warmth, asymmetry, and slight imperfection — avoid the glossy 'AI render' look.
- **Optimisation:** WebP/AVIF, `loading="lazy"`, explicit `width`/`height` to prevent layout shift.
- **Icons:** `lucide-react` for UI. Use sparingly — never decorative-only.
### Build-time asset manifest (explicit specs)
Every image, illustration, and visual reference mentioned above must resolve to ONE of the three buckets below — runtime-generated, seed-shipped, or user-supplied. Do NOT ship `` tags whose `src` is not listed here. Do NOT depend on bare "section 8a prompts" without binding them to explicit paths and model IDs.
**Bucket 1 — Runtime-generated (Nano Banana Pro `gemini-3-pro-image` for hero/demo photographs; Nano Banana 2 `gemini-3.1-flash-image` for in-app illustrations and reference-conditioned variants).** Cached to Firebase Storage; served via signed URL. Every reference above to "Nano Banana 2" or "Nano Banana Pro" MUST be wired to one of these specific calls with an explicit model id:
- `/public/generated/hero.webp` (2400×1500, WebP) — model `gemini-3-pro-image` — uses the literal prompt described as "Hero image (landing screen)" above. Run once at build; commit a `/public/samples/hero-fallback.webp` (1600×1000) generated from the same prompt with `gemini-3.1-flash-image` so the page renders if quota is exhausted.
- `/public/generated/demo/{demo-slug}-{NN}.webp` (1600×1200, WebP) — model `gemini-3.1-flash-image` (reference-conditioned where the prior frame is passed as input) — one path per "Demo X" image referenced above. The slug derives from the seed example in section 8a; the NN index covers each frame in the demo sequence.
- `/public/generated/illustrations/{name}.webp` (1024×1024, WebP) — model `gemini-3.1-flash-image` — one path per named illustration above ("Empty-state illustration", "Recipe-card hero illustrations", "Curriculum picker imagery", "Period-style frames", etc.). Each illustration's prompt is the literal description above; ship a deterministic seed in the request so re-runs are reproducible.
**Bucket 2 — Seed assets shipped with the deliverable.** Every "Stock fallback" path referenced above (e.g. `/public/samples/sample-X.jpg`) is generated once via Nano Banana 2 (`gemini-3.1-flash-image`) at 1024×1024 WebP using the same prompt as its Bucket-1 counterpart, then committed to the repo so the page renders identically if Gemini quota is exhausted or the user is offline. Replace any `.jpg` extension above with `.webp` to match the optimisation rule. Also commit these empty-state seeds (1024×1024 WebP, single-stroke hand-drawn line, no colour fill):
- `/public/samples/empty-state-primary.webp` — line drawing of the app's primary empty surface (the named "Empty-state illustration" above), generated from that exact prompt.
- `/public/samples/empty-state-archive.webp` — line drawing of an empty saved/archive view, single-stroke outline.
- `/public/samples/empty-state-error.webp` — line drawing of a hand placing a single object aside with care, used when an AI call fails.
**Bucket 3 — User-supplied.** Uploads from the user's camera / file picker land at the Firebase Storage path conventional for this template (named in section 4b). The build ships with Bucket-1 + Bucket-2 only; no user-supplied images at first paint.
**Hard rules**
- Every `` tag MUST have a `src` that resolves to a path listed in Bucket 1, Bucket 2, or a Bucket 3 upload path. Anything else is a build error.
- No bare `image.jpg` / `hero.jpg` / `placeholder.png` references anywhere in the code.
- Model IDs: `gemini-3-pro-image` for hero-quality photographic generation; `gemini-3.1-flash-image` for in-app illustrations, reference-conditioned variants, empty-state seeds, and stock fallbacks. Never use a legacy model id (no `imagen-*`, no `gemini-1.5-*-image`).
- File format: WebP everywhere (AVIF acceptable where the target browsers support it). No `.jpg` / `.jpeg` / `.png` in `/public/samples/`.
## 10. Interactivity & states
- Every interactive element has hover, focus, active, and disabled states.
- Forms validate inline and show specific error messages (not "Invalid input").
- Loading states use skeletons that match the eventual yarn-card layout, not spinners.
- Empty states explain the next action with a button whose label fits THIS app's domain: "Photograph the basket", "Hold one new skein up with the label facing the camera", "Invite a knit-night friend" — never a generic "Add your first item".
- Smooth scroll for in-page anchors.
- All AI-generated content streams in token-by-token where supported, with a clear "thinking…" indicator before content starts arriving.
- If an AI call fails, show a calm, specific error ("We couldn't pick out individual balls in this photo — the basket may be too densely packed; want to try a clearer wide-shot, or photograph one ball at a time?") and offer retry.
- Low-confidence fibre estimates are faintly underlined; tapping reveals the alternate fibres the model considered, alongside the confidence value.
- The Colour view's hue-shuffle when a new entry is added takes 500 ms with `prefers-reduced-motion` falling back to instant.
- The "(i) how the app made this guess" expandable panel slides open with a thin shadow; reduced-motion users see it appear instantly.
- The confidence chip is colour-coded: indigo for ≥ 0.9 (label-read), neutral for 0.7-0.89 (vision-confident), madder-red for below 0.7 (guess). Hover reveals the calibration scale.
## 11. Tech & responsive requirements
- **TTS markdown-stripping preprocessor:** before sending any user-authored markdown to `gemini-3.1-flash-tts-preview`, strip non-spoken markdown: `#`/`##`/`###` headings (keep the title text), `**bold**` (keep the inner text), `[label](url)` (keep `label`, drop URL), `` ``` `` fenced code blocks (skip entirely), `>` block-quote markers (keep the text), and `|` table pipes (read row-by-row as sentences). Insert `…` between sentences for a short pause and a blank line plus `—` between paragraphs for a long pause. The model does not understand markdown; raw markdown will be read aloud as literal characters ("asterisk asterisk").
- **File downloads on Safari / Firefox:** when offering local-disk save of any export (PDF, CSV, MP3, ZIP, JSON, image), fall back to `` with a blob URL — the File System Access API (`showSaveFilePicker()`) is Chromium-only. Detect with `'showSaveFilePicker' in window`; otherwise use the anchor-download path.
- **Stack:** React + TypeScript + Tailwind CSS. Functional components + hooks. Use Shadcn UI primitives where appropriate.
- **Build runtime:** AI Studio Build — full-stack with Cloud Run server-side functions. All Gemini API calls happen server-side; API key lives in Secrets Manager, never in client bundle.
- **Model selection:** explicitly pin `gemini-3.5-flash` for the basket-parse and the whole-stash project planner; `gemini-3.5-flash` for colour extraction, per-yarn project suggestion, and label-OCR-only intake; `gemini-3.1-flash-tts-preview` for narration; `gemini-3.1-flash-image` for the empty-state illustration. Set `thinkingLevel` explicitly per call. Omit `thinkingConfig` entirely on TTS and image-generation calls.
- **Database:** Firestore (auto-provisioned by AI Studio Build). Show the seed stash on first launch.
- **Auth:** Firebase Auth — Google sign-in by default; Apple sign-in next to it (requires Apple Developer account + Service ID config); magic-link email as fallback (requires sender-domain authorisation).
- **Storage:** Firebase Storage for original photographs and per-ball thumbnails. **Not auto-provisioned by AI Studio Build** — enable in the Firebase console and wire the bucket name into the project before first photograph upload. Pre-signed URLs only.
- **Files API for Gemini:** photographs uploaded to Gemini are passed via the Files API (`files/*` resource name) or `inlineData` (base64). Do NOT pass Firebase Storage public URLs directly to `generateContent` — the API does not fetch them server-side.
- **Long-context guardrail:** the whole-stash project planner call is chunked by weight class or fibre family when the stash exceeds 1,200 entries, to stay safely under the 1M token ceiling.
- **Mobile-first.** Verify layouts at 375 px (iPhone SE), 768 px (iPad), 1024 px, 1440 px+.
- Use `clamp()` for fluid typography. Prefer container queries over media queries for component-level responsiveness.
- Use `dvh` / `svh` instead of `vh`. Respect safe-area insets on iOS.
- Zero horizontal overflow at any width. Zero layout shift on load.
- Persist user data in Firestore. Use real-time listeners on the stash view.
- Optimistic UI on writes; reconcile on response.
- Wide-shot capture flow uses the Web Camera API with fixed focus/exposure where supported; falls back to native camera otherwise.
- **iOS Safari gotchas (graceful degradation):** camera permission does NOT persist across page reloads on iOS — re-request on every basket-scan session; backgrounded Safari tabs pause `getUserMedia` — re-acquire the stream on `visibilitychange`; on Low Power Mode iOS may degrade resolution — always offer `` as a fallback so a basket photo still uploads when WebRTC is denied; rotation drops the camera track on iOS — re-bind on `orientationchange`.
## 12. Accessibility (WCAG 2.2 AA)
- Semantic HTML — `header`, `nav`, `main`, `section`, `article`, `footer`.
- All interactive controls reachable by keyboard with a visible focus ring.
- Color contrast ≥ 4.5:1 for body, 3:1 for large text and UI components. Verify that the indigo confidence chip on the undyed-wool background meets contrast; verify that the madder-red low-confidence chip meets contrast.
- All images have meaningful `alt` text. The original photographs have `alt` describing the artefact ("wide-shot of an open basket of yarn on a rug, 32 visible balls in mixed colours, afternoon window light").
- Form fields have associated `