================ ONE-SHOT BUILD CONTRACT (read first) ================
Build this in Google AI Studio "Build" in ONE shot — a complete, working app,
no follow-up turns. These are hard rules, not suggestions:
1. TARGET = Full-Stack Web (Node server runtime, secrets, Firebase allowed).
If you target Native Android instead, you MUST drop all server/DB/Workspace/
secrets and persist locally (Room / SharedPreferences) only.
2. PIN EVERY MODEL CALL — never let the agent auto-select (it downgrades on quota):
- Reasoning / text -> gemini-3.5-flash (thinkingLevel: minimal|low|medium|high)
- 4K image + legible text -> gemini-3-pro-image (image_size "4K", up to 14 refs)
- High-volume image -> gemini-3.1-flash-image
- Expressive TTS -> gemini-3.1-flash-tts-preview (inline tags e.g. [whispers])
- Realtime audio/video (WebSocket) -> gemini-3.1-flash-live-preview
- Sandboxed agent -> antigravity-preview-05-2026
3. DIVISION OF LABOR — the model ONLY parses/extracts to a strict responseSchema.
ALL math, money (store currency as integer minor units / cents), sorting,
balancing and graph logic run in deterministic TypeScript/Python. The model
must never compute totals, splits or balances itself.
4. responseSchema sanitation — no regex patterns, no fixed-length tuples, no
format validators in the schema (they crash the OpenAPI engine). Enforce those
in server-side code AFTER parsing the JSON.
5. responseSchema and google_search grounding are MUTUALLY EXCLUSIVE in one call.
6. CODEGEN — split large output into modular, single-responsibility files so no
file is truncated by the output-token cap.
7. Every external call gets a graceful fallback (e.g. manual paste if a Workspace
read fails). Never a silent dead end.
8. ROBUST STORAGE & CANVAS — Wrap all `localStorage`/`sessionStorage` operations (especially JSON parsing and writes) in `try-catch` blocks to prevent crashes in private windows or quota overflows. Canvas drawing elements must dynamically handle window resize and scale pixel density (`window.devicePixelRatio`) to avoid blurry graphics on retina displays.
=====================================================================
# MUST OBEY — Mobile-first build requirements
This app's PRIMARY surface is a mobile phone. Build it impeccably on mobile FIRST, then verify on tablet and desktop. Treat the rules below as non-negotiable hard constraints, not suggestions.
## Viewports to verify (every screen, every state)
- 320 px, 360 px, 375 px, 390 px, 414 px, 480 px
- 768 px, 834 px (iPad portrait / Pro 11)
- 1024 px, 1280 px, 1440 px, 1920 px, 2560 px
- Plus: 200% browser zoom, landscape orientation on every mobile width, iPhone with safe-area insets visible
## Hard layout rules
- Mobile-first CSS. Default styles target mobile; `@media (min-width: ...)` for larger viewports.
- Use `dvh` and `svh` instead of `vh` for full-height surfaces (iOS Safari URL-bar bug).
- Use `clamp()` for fluid typography across all viewports.
- Prefer container queries (`@container`) over media queries for component-level responsiveness.
- Use `min(100%, ...)` widths so content never overflows. Zero horizontal overflow at any viewport.
- Add `` to every page.
- Apply `padding: max(safe-area-inset-X, fallback)` on every edge-bleeding container so notched iPhones in landscape never clip content.
- Wide tables and code blocks scroll INSIDE their container (`overflow-x: auto`), never push the body.
- Use `background-attachment: scroll` on mobile, not `fixed` (iOS Safari repaint bug).
- Avoid `backdrop-filter` on animated elements. Use it sparingly on static surfaces only.
- **Canvas Scaling**: Canvases must dynamically scale with window resize events and properly handle high-DPI screens (`window.devicePixelRatio`). Set physical dimensions (`canvas.width`/`canvas.height`) using pixel ratio and render relative to this grid, using CSS to control responsive viewport scaling.
- **Robust Storage**: Every access to `localStorage`/`sessionStorage` (especially `JSON.parse` of loaded state or writes) MUST be wrapped in a `try-catch` block to handle disabled storage, private browsing mode, quota limits, or corrupted JSON gracefully. Fall back to a robust in-memory object store.
## Touch & accessibility
- Tap targets ≥ 44 × 44 px on touch (Apple HIG). Increase to 48 px under `@media (hover: none) and (pointer: coarse)`.
- All interactive controls reachable by keyboard with a visible focus ring; respect `:focus-visible`.
- Color contrast ≥ 4.5:1 for body text, 3:1 for UI components.
- All images have meaningful `alt`. Decorative images use `alt=""`.
- Respect `prefers-reduced-motion: reduce` — zero animation durations under that query.
- Forms validate inline; error messages are specific, not "Invalid input".
- Modals: focus trap, `Esc` closes, `role="dialog"`, `aria-modal="true"`, focus restored on close.
## Performance bar (Lighthouse mobile, throttled 3G/4G)
- LCP < 2.5 s · INP < 200 ms · CLS < 0.1
- JS bundle gzip < 200 KB mobile-first; lazy-load non-critical screens via `React.lazy` / dynamic imports.
- No render-blocking resources above the fold.
- Images: WebP/AVIF preferred, `loading="lazy"`, explicit `width`/`height` attributes (zero CLS), `srcset` for retina.
- Videos: `preload="metadata"`, low-resolution poster, max 720p mobile fallback. Never autoplay with audio.
- Fonts: `font-display: swap`; preload only the one used above the fold.
- Smooth scroll honoured via CSS `scroll-behavior: smooth` with reduced-motion fallback.
## Pre-ship mobile checklist (the deployer MUST verify before declaring done)
1. Open at 375 px in DevTools — every screen scrolls vertically only; zero horizontal scroll.
2. Browser zoom 200% — layout reflows without overlap.
3. iPhone Safari with the URL bar visible AND landscape — no content under the home indicator; no notch clipping.
4. iPad portrait (768 px) and landscape (1024 px) — no awkward gaps; tablet-specific breakpoints land cleanly.
5. Tap every interactive element with a thumb at real-device size — every target is easy to hit.
6. `prefers-reduced-motion: reduce` — every transition / animation skips cleanly, scroll-behavior becomes instant.
7. Lighthouse mobile score ≥ 90 across all 4 categories.
8. Zero `console.error` and zero CLS shift in real-device testing on a mid-tier Android (e.g. Pixel 6a) and an iPhone SE.
---
The original template starts below. All rules above apply on TOP of whatever this template specifies.
---
# Lecture, Condensed
## 1. Project
**Lecture, Condensed** is a study tool for people who have more lecture
video than they have time. The user drops in the URLs of nine hours of
recorded lectures (the whole semester's worth, the conference playlist,
the bootcamp recordings, the cardiology series the registrar saved for
the night before the exam), and the app produces a structured study
set: a tight outline of every concept introduced, the exact verbatim
quote from the lecturer where each claim is made, the timestamp of
that moment in the video, a deck of spaced-repetition cards keyed to
those quotes, and — the bit users come back for — a short list of
"the seven minutes you actually need to re-watch tonight". Every card,
every claim, every line of the outline cites a timestamp. The app
never invents content that is not in the lecture.
This is the kind of app a medical student in Boston builds at 11pm
the night before her cardiology shelf exam, with the recordings of
nine first-year lectures she half-attended on her tablet, a cold cup
of chai on the desk, and a 9am call time in the morning. It is also
the kind of app a PhD student in Lisbon turns to a week before her
qualifying exam — twelve hours of econometrics lectures in Portuguese
on her supervisor's department channel, an inflexible exam date, a
toddler asleep in the next room. Same shape of moment, different
field, different language.
The single demo that proves the magic: drop the URL of a ninety-minute
lecture (Khan Academy, MIT OpenCourseWare, a Coursera video, an
unlisted YouTube link of a captured Zoom recording) → in under five
minutes the user sees a structured outline with every section
timestamped, sixty Anki-compatible flashcards with verbatim
lecturer-quote justifications on the back of each card, and a
"re-watch list" that picks out three short windows — together about
seven minutes — that the model identifies as the parts where the
lecturer slowed down, introduced a definition, or said "this is
exactly the kind of thing that comes up on the exam".
And in the harder cases — non-English lectures, low-quality
recordings, multi-speaker panels, lectures that drift, lectures
where the slides are the substance and the speech is filler — the
app handles the long-context messiness: it transcribes in the
source language and translates to the user's language, attributes
quotes to the right speaker on a multi-speaker panel, distinguishes
"this is on the exam" emphasis from rhetorical asides, and never
silently smooths a stutter or omission that might matter.
**Tagline:** _Turn nine hours of lecture video into the seven minutes you actually need to re-watch — in any subject, any language, with every claim timestamped to the second._
## 2. Target audience
- Graduate and professional students with semester-long lecture
series to revise before high-stakes exams — medical (USMLE shelf,
PLAB, NEET-PG, KMLE), law, finance certifications, qualifying
exams, comprehensive exams
- Undergraduate students who missed lectures and are working from
the captured Zoom or Panopto recording the day before a midterm
- Bootcamp learners working through 60+ hour recorded curricula
(Lambda School, Le Wagon, Codecademy Pro, General Assembly,
YouTube lecture series from Andrej Karpathy or Stanford CS231n)
- Adult learners auditing university courses — retirees taking
MIT OCW Linear Algebra, parents auditing the toddler-asleep hours
of MOOC content
- Researchers consuming conference proceedings (NeurIPS, ICML,
ACL, AAAS, ASCO, ESC) where the talks are on YouTube and the
attendee has a flight tomorrow
- Continuing-medical-education and CPD learners — nurses, doctors,
pharmacists, lawyers, accountants — who must consume recorded
modules before a deadline and want structured notes for re-reference
- ESL / EAL learners who can follow a lecture in English but want
a second pass with verbatim transcripts and translated explanations
in Tagalog, Mandarin, Korean, Vietnamese, Tamil, Hindi, Urdu,
Bengali, Punjabi, Amharic, Swahili, Farsi, Khmer, Portuguese,
or Spanish
- Educators preparing summaries of their own recorded lectures to
share with students who missed class, or to seed a study guide
- High-school students preparing for AP exams, A-Levels, IB,
Brazilian ENEM, Indian JEE, Korean Suneung, Japanese Center Test
— the captured YouTube tutoring series is the textbook
- Professional musicians and dancers studying recorded masterclasses
(a violinist learning from a Heifetz masterclass YouTube channel,
a choreographer studying repertory video from the Wuppertal archive)
## 3. Core value propositions
Surface these clearly through copy, visual emphasis, and section ordering — they are the reasons users pick this app.
- **Watches every minute so you don't have to** — Gemini 3.5 Flash's
native long-context video understanding reads a full ninety-minute
lecture in one pass. No splitting into chunks that lose context;
no audio-only transcript that drops the slide content. The model
sees the speaker, hears the speech, reads the slides, and notices
when the slide changes ahead of the speech.
- **Every claim is timestamped, every quote is verbatim** — the
outline does not summarise into the model's voice. Each bullet
cites the lecturer's exact words and the second of the lecture
where they were said. A claim without a timestamp does not appear.
- **The seven-minute re-watch is the killer feature** — instead of
re-watching the whole lecture, the user gets a ranked list of
short windows (typically 60-180 seconds each) where the lecturer
introduced a key definition, worked a critical example, signalled
that something would be on the exam, or showed a slide too complex
to absorb in real time.
- **Anki cards keyed to the source** — each card's back includes
the verbatim lecturer quote and a timestamp link that opens the
source video at the exact second. Export to `.apkg` (Anki) or
`.csv` (Quizlet, Brainscape, RemNote) or print as flashcard PDF.
- **Cites the lecturer, not the model** — the app never invents
content that is not in the lecture. If the user asks "did the
lecturer say anything about X?" and the lecturer did not, the
app says so. The model is constrained to be a faithful reader,
not a co-author.
- **Reads non-English lectures faithfully** — Tagalog, Vietnamese,
Mandarin, Cantonese, Korean, Tamil, Hindi, Urdu, Bengali, Punjabi,
Amharic, Swahili, Farsi, Khmer, Portuguese, Spanish, French,
German, Russian, Polish, Turkish, Indonesian, Thai. Transcript
stays in the source language. Translation runs on a second pass,
preserving technical terminology verbatim where appropriate.
- **Handles multi-speaker panels** — conference recordings,
symposia, journal-club discussions: the app attributes each quote
to the right speaker (by voice + visual + slide-author attribution
when on screen). The notes show "(Dr Patel, panellist)" not
"(speaker 2)".
- **Survives bad recordings** — the auditorium mic that picks up
the front row but not the back, the Zoom recording where the
presenter forgot to share audio, the lecture where the laser
pointer is the only annotation. The app flags low-confidence
passages so the user knows where to listen carefully.
- **Privacy** — the user's recordings (even unlisted YouTube links,
even self-uploaded files) are not used to train any model. This
matters when the lecture is a faculty-internal recording, a
paid course, or a private institutional video.
## 4. Features to build
- URL paste box accepting one URL or many (one per line), with
drag-and-drop support for `.mp4`, `.mov`, `.webm` files uploaded
directly to the user's Firebase Storage bucket
- Source-aware ingestion — YouTube (public + unlisted), Vimeo,
direct file upload via Files API, Panopto / Kaltura with
user-supplied auth headers, MIT OpenCourseWare media URLs
- Per-video long-context call to Gemini 3.5 Flash with the full video
as input, returning the structured `LectureNotes` schema below
- Outline view — chronological by timestamp, collapsible by section,
with a "[show original quote]" expander next to each bullet that
reveals the verbatim line from the lecturer
- Anki-compatible flashcard generation with verbatim back-of-card
quotes, mapped concepts, and timestamp deep-links to the source
video; export `.apkg` directly (genanki-compatible field order)
- "Re-watch in seven minutes" — ranked windows view with auto-play
through the shortlist, total elapsed counter, and a one-tap
"watched, send me the next" button
- Multi-video archive — a "course" object groups many videos, and
cards / outline / re-watch lists span the whole course
- Cross-lecture concept linking — if the same concept (the
Hodgkin-Huxley model, the Black-Scholes equation, the
Krebs cycle, the Polish vowel system, the Ginzburg-Landau
theory) appears in two lectures, the People-and-concepts view
links them
- Search across the course — semantic ("show me everywhere the
lecturer talked about partial differential equations") and
structured ("from week 4, mentioning eigenvalues")
- "Did the lecturer say…?" answer box — strict mode that returns
yes / no / no-clear-answer, with the timestamp of the closest
matching passage if any
- Quiz mode — the app generates a 10-minute self-test from the
selected lecture's flashcards, scores it, and produces a focused
re-watch list of just the cards the user missed
- Speaker attribution toggle — single-speaker mode (lecture),
multi-speaker mode (panel, journal club, Q&A), auto-detect mode
- Slide-text extraction — slide content is read as visual data,
not transcribed from speech; a slide that says "P(B|A) = P(A|B)P(B)/P(A)"
is captured as that exact formula, not as the lecturer's
spoken paraphrase
- Equation rendering — formulas in transcript and slides are
rendered with KaTeX in the outline view; the raw LaTeX string
is preserved in the structured output
- Diagram description — if the lecturer draws a diagram on the
whiteboard or shows a figure on a slide, the diagram is captured
as a sentence-level description plus an image crop, never silently
dropped from the notes
- Time-window pinning — the user can mark a 30-second window
("I need to re-watch this specifically") and the window appears
at the top of their re-watch list regardless of model ranking
- Bilingual notes — for non-English lectures, side-by-side source
transcript and translated outline; toggle to switch the
flashcard front-language for spaced repetition
- Export to PDF (study handout) and `.md` (Obsidian / Roam
compatible, with `[[timestamp]]` deep links)
- Privacy-first sharing — a course can be shared read-only with a
classmate via magic-link; cards and outline are visible but the
source video URLs are gated (the classmate clicks through to
YouTube directly)
- Honest "we couldn't" reporting — if the audio is too poor for a
passage to be reliably transcribed, the app flags the window
with a "low confidence — listen yourself" badge rather than
guessing
## 4b. Required Gemini capabilities + backend services
**This template's intelligence comes from the Gemini capabilities below. Wire them up explicitly — don't substitute generic LLM calls.**
### Gemini capabilities (the load-bearing intelligence)
- **Long-context video input** (Gemini 3.5 Flash) — the hero capability.
Reads a full ninety-minute lecture video in a single call: speech,
speaker visual, slide content, on-screen text, whiteboard writing,
laser-pointer emphasis, the moment the lecturer says "this is
exam-relevant". One call per video, not per chunk. Video is
uploaded via the Gemini Files API (`files/*` resource name).
- **Structured output / JSON Schema** — the response matches the
`LectureNotes` schema below. Every field is typed; the schema is
included verbatim in the system instruction and as `responseSchema`.
- **Multilingual transcription + translation** (built into Gemini 3
Pro) — handles the lecture's source language directly without a
separate Whisper call; transcript stays in the source language;
translation produced as a second-pass call when needed.
- **Long context (1M tokens)** — the per-video call typically fits
in 200-400k tokens; the cross-lecture / course-wide concept-linking
call may approach the 1M ceiling for a 20-video course. **Guardrail:**
a parsed `LectureNotes` object averages ~3,000 tokens of structured
output plus ~30,000 tokens of input per ninety-minute video; a
course of 12 videos ≈ ~400k tokens (comfortable). For courses
larger than 20 ninety-minute videos, chunk the cross-lecture call
by topic-cluster or by week before invoking — the 1M ceiling is
real and a 30-video course will exceed it.
- **Thinking levels** — `medium` for the primary per-video parse
(the model has to weigh which moments are exam-relevant, which are
asides, which slides are load-bearing). `low` for translation,
flashcard generation, and the "did the lecturer say X?" yes/no
call. Surface `thoughtSummary` only when the user clicks the
small "(i) show how the AI chose this re-watch window" icon next
to a re-watch entry.
- **Gemini TTS** (`gemini-3.1-flash-tts-preview`) — optional
audio-summary mode: the model reads the outline aloud at the
user's chosen pace, in the user's chosen language, while the
user commutes. Source-language playback is supported (Korean
lecture summarised in Korean for the Korean-speaking student;
same lecture summarised in English for the English-speaking
classmate sharing the course).
### Backend services
- **Auth — Required.** Firebase Auth with Google sign-in
(auto-provisioned by AI Studio Build). **Apple sign-in is optional
but user-configured**: it requires an Apple Developer account,
Service ID, Key ID, and private key wired into the Firebase Auth
console. **Magic-link email** (used for sharing a course with a
classmate) requires the sender domain to be authorised in Firebase
Auth. Courses are private to the owner and explicitly-shared
classmates. No public-by-default.
- **Database — Required.** Firestore for `users`, `courses`,
`videos`, `lecture_notes`, `cards`, `rewatch_windows`,
`course_members`.
- **File storage — Required.** Firebase Storage for user-uploaded
video files (when the lecture is not on YouTube) + processed
thumbnails + slide-frame crops. **Storage is NOT auto-provisioned
by AI Studio Build today** — enable it in the Firebase console
and wire the bucket name into the AIS Build project before first
video upload. Pre-signed URLs only; the video files are never
publicly addressable.
- **Email — Required (transactional).** Course-sharing invitations
via email link (Firebase Auth magic links). Exam-eve digest
emails ("here's tomorrow's quiz from the re-watch list") if the
user opts in.
- **Payments — Not needed for v1.** Free for personal use. A future
"classroom tier" could meter long-context API spend per teacher
account.
- **External APIs:** Gemini API for all intelligence; `yt-dlp` or
the YouTube Data API (server-side, with explicit user consent
for the URL) for fetching publicly-listed metadata. **Do NOT
bypass YouTube paywalls or download private videos without the
uploader's consent.** Public + unlisted-with-user-supplied-link
only.
**Environment variables:** every secret (Gemini API key, Firebase
service-account JSON, YouTube Data API key if used) lives in
environment variables — never in client bundle. Include a
`.env.example`.
**Auth + data privacy reminders:** never log secrets · never store
passwords in plain text · use HTTPS everywhere · honour 'delete my
account' inside the UI · explicit opt-in for any analytics · the
user's recordings are never sent to Gemini for model training (use
the Gemini API on the paid tier, where Google does not use your
content for model training, per the Gemini API Additional Terms) ·
the user's flashcards, notes, and re-watch lists are private to
their account and the classmates they explicitly share with ·
unlisted YouTube URLs are treated as confidential and never
indexed, never appear in any public surface.
**Read this first — prompt-craft rules that apply to every call in this template:**
1. **Name the model variant explicitly** in every Gemini API call.
2. **Grounded responses can wrap JSON in ```json fences or add prose preamble.** Server-side, strip fences and brace-extract:
```typescript
function safeExtractJSON(raw: string): T {
const clean = raw.replace(/```json\s*|```/gi, '').trim();
const s = clean.indexOf('{'); const e = clean.lastIndexOf('}');
if (s === -1 || e === -1) throw new Error('No JSON boundaries in grounded response');
return JSON.parse(clean.slice(s, e + 1)) as T;
}
```
3. **Strip unsupported Zod modifiers before passing to `responseSchema`** — Gemini's OpenAPI subset rejects `.regex()` / `pattern`, fixed-length `z.tuple()`, and other custom validators. Use a sanitizer that flattens tuples to length-2 arrays and removes regex patterns before serializing. Validate those constraints in middleware AFTER parsing.
Do not let the agent pick the model. See the per-call matrix below.
2. **Pin `thinkingLevel` explicitly** per call. See the matrix.
3. **Seed the JSON Schema as a fenced TypeScript / Zod block** in
the system instruction or `responseSchema` field. The literal
schema is below. **Convert the Zod schema to Gemini's `Schema`
type via the SDK helper** before passing to `responseSchema` —
do NOT pass raw Zod. **Numeric `min`/`max` constraints are
documentation only inside `responseSchema`; clamp on the server
after the response arrives.**
4. **Pin the system instruction separately** from user input. Use
the `systemInstruction` field for persona + behavioural rules;
use `contents` for user input (the video file reference). Never
concatenate.
5. **Pre-declare tools as an enable/disable list** per call. The
matrix below names which tools are enabled per call. Tools NOT
listed for a call should be disabled.
6. **State negative constraints explicitly** — they are listed
below. They are NOT "be careful" suggestions; they are hard
rules the model must follow.
### Per-call model + tools matrix
| Call | Model | thinkingLevel | Tools enabled |
|------|-------|---------------|---------------|
| Watch lecture video, parse → `LectureNotes` schema | `gemini-3.5-flash` | medium | (none) |
| Translate transcript + outline to target language | `gemini-3.5-flash` | low | (none) |
| Generate Anki flashcards from `LectureNotes` | `gemini-3.5-flash` | low | (none) |
| Rank re-watch windows from `LectureNotes` | `gemini-3.5-flash` | medium | (none) |
| Course-wide cross-lecture concept linking | `gemini-3.5-flash` | medium | (none) — long-context over the course |
| "Did the lecturer say X?" strict-quote answer | `gemini-3.5-flash` | low | (none) |
| Generate TTS audio summary in source/target language | `gemini-3.1-flash-tts-preview` | n/a | n/a |
| Generate slide-frame crop description | `gemini-3.5-flash` | low | (none) |
*Note for builders:* on TTS and image-generation calls, omit
`thinkingConfig` entirely — the field is not supported on those
models. The `n/a` cells in this matrix are documentation only; do
not serialise them into the request body. The schema-bearing calls
above never combine `responseSchema` with `google_search`
grounding; this template does not need grounding because the
source of truth is the lecture video itself, not the open web.
### Primary structured-output schema (seed this verbatim in the prompt)
```typescript
import { z } from "zod";
const Timestamp = z.object({
seconds: z.number().min(0),
hhmmss: z.string(), // "01:23:45" — display
});
const TimeWindow = z.object({
start: Timestamp,
end: Timestamp,
duration_seconds: z.number().min(0),
});
const SpeakerAttribution = z.object({
speaker_label: z.string(), // "Dr Patel" if known, otherwise "lecturer", "panellist-1"
inferred_role: z.string().nullable(), // "primary lecturer", "panellist", "questioner from audience"
attribution_confidence: z.number().min(0).max(1),
attribution_basis: z.array(z.enum([
"name_on_slide", "voice_consistency", "visible_on_camera",
"introduced_by_name", "title_card", "unknown",
])),
});
const VerbatimQuote = z.object({
text: z.string(), // exact lecturer words, no smoothing
timestamp: Timestamp,
source_language: z.string(), // BCP-47, "en-US", "pt-BR", "ko-KR"
speaker: SpeakerAttribution,
is_partial_or_inaudible: z.boolean(),
inaudible_note: z.string().nullable(), // "second half of sentence inaudible — auditorium noise"
});
const SlideCapture = z.object({
appeared_at: Timestamp,
visible_until: Timestamp,
slide_text_verbatim: z.string(), // exact text on slide, including formulas
contains_equation: z.boolean(),
latex_strings: z.array(z.string()), // each formula as LaTeX
contains_diagram: z.boolean(),
diagram_description: z.string().nullable(), // one sentence, sensory
is_load_bearing: z.boolean(), // model's judgement: this slide is the substance, not chrome
});
const OutlineEntry = z.object({
section_title: z.string(), // model's structural title — short, faithful
section_window: TimeWindow,
bullets: z.array(z.object({
point: z.string(), // the claim, in plain language
supporting_quote: VerbatimQuote, // verbatim line from the lecturer
related_slide_index: z.number().nullable(), // points to SlideCapture[]
confidence: z.number().min(0).max(1),
})),
is_definition: z.boolean(),
is_exam_signalled: z.boolean(), // lecturer said "this is on the exam" or equivalent
is_worked_example: z.boolean(),
});
const RewatchWindow = z.object({
window: TimeWindow,
rank: z.number().min(0), // 0 = highest priority
reason: z.enum([
"key_definition_introduced",
"worked_example",
"exam_signalled",
"slide_too_dense_for_realtime",
"concept_that_recurs_later",
"lecturer_slowed_down",
"common_misconception_addressed",
"user_pinned",
]),
one_line_justification: z.string(), // "lecturer says 'this exact form will appear on the exam'"
supporting_quote: VerbatimQuote,
});
const Flashcard = z.object({
card_id: z.string(),
front: z.string(), // question or prompt
back: z.string(), // answer in the user's words OR the lecturer's words
back_is_verbatim_quote: z.boolean(), // true if the back is the lecturer's exact words
supporting_quote: VerbatimQuote, // always present, always cited
related_outline_section: z.string().nullable(), // OutlineEntry.section_title
related_concept_tags: z.array(z.string()), // ["Hodgkin-Huxley", "action potential", "ion channels"]
difficulty: z.enum(["recall", "understand", "apply", "synthesise"]),
});
const ConceptMention = z.object({
concept_name: z.string(), // "Hodgkin-Huxley model"
context_quote: VerbatimQuote,
is_introduced_here: z.boolean(), // first time in the course it appears
is_revisited_here: z.boolean(), // referred back to from an earlier mention
});
const LectureNotes = z.object({
video_id: z.string(), // app's internal id
source_url_or_file_ref: z.string(),
duration_seconds: z.number().min(0),
source_language: z.string(), // BCP-47
source_language_confidence: z.number().min(0).max(1),
detected_speakers: z.array(SpeakerAttribution),
primary_speaker_label: z.string(),
topic_one_line: z.string(), // "the Hodgkin-Huxley model of the action potential"
topic_long_form: z.string(), // 2-4 sentence summary, faithful, no embellishment
outline: z.array(OutlineEntry),
slides: z.array(SlideCapture),
concepts_mentioned: z.array(ConceptMention),
rewatch_windows: z.array(RewatchWindow), // server filters to top-N for the seven-minute list
flashcards: z.array(Flashcard), // typically 30-90 per ninety-minute lecture
overall_reading_confidence: z.number().min(0).max(1),
audio_quality_note: z.string().nullable(), // "lecturer near mic, audience-question audio poor"
flagged_for_user_review: z.array(z.object({
field_path: z.string(), // "outline[3].bullets[2]"
reason: z.string(),
})),
});
type LectureNotes = z.infer;
```
### Common failure modes (and how to avoid them)
- Agent picks `gemini-3.5-flash` for the parse to save quota — pin
`gemini-3.5-flash` explicitly. Flash drops half the slide content,
conflates the lecturer with audience-question speakers, and
hallucinates timestamps that drift by 30+ seconds. The hero
capability is *long-context video*; Pro is non-negotiable.
- Model invents content not in the lecture — the most common
failure. The system instruction must be unambiguous: every bullet
must be backed by a verbatim quote with a timestamp; bullets
without a quote do not appear. Add a server-side validator that
rejects any `OutlineEntry.bullets[i]` whose `supporting_quote.text`
is empty.
- Timestamps drift by 5-20 seconds — the model sometimes anchors
to slide changes instead of speech onset. The system instruction
says "the timestamp is the second at which the lecturer began
saying the quoted sentence, not the second the related slide
appeared". Server-side: compare quoted text against the
lecture's audio transcription window; reject if delta > 5
seconds.
- Re-watch windows pick the entire 90 minutes — the model defaults
to "include everything important". The system instruction caps
the total combined duration at 8 minutes across all windows
(typically 3-5 windows). Server-side: validate total duration;
if exceeded, re-rank by `rank` and trim.
- Slide formulas captured as paraphrase instead of verbatim — the
model may convert "P(B|A) = P(A|B)P(B)/P(A)" to "the conditional
probability formula". Hard rule: every formula on a slide is
captured as the literal characters on the slide, AND as a LaTeX
string in `latex_strings[]`.
- Multi-speaker panels collapse to single speaker — the model
attributes every quote to "the lecturer". Pin the system
instruction: "if a second person speaks for more than 30
consecutive seconds, treat them as a distinct speaker with their
own attribution".
- Non-English lectures summarised in English instead of source —
the transcript / outline should stay in the source language by
default; translation is a *separate* call. Don't conflate.
- Anki cards generated without verbatim quotes — the cards are
the most-shared output and the most likely to circulate. Every
card must carry the lecturer's verbatim quote on the back, with
the timestamp. Server-side: reject any flashcard whose
`supporting_quote.text` is empty.
- TTS reads the lecturer's name with the wrong pronunciation — the
TTS model is not given the speaker's name; it is given the
outline text. Source-language voice locale is picked via
`languageCode`. Pronunciation of proper names is best-effort;
the speaker's actual recorded pronunciation is the source of
truth (link to it).
- Course-wide concept linking exceeds 1M tokens — chunk by
topic-cluster or by week. Each chunk produces partial
`ConceptMention[]`; merge server-side. Do not send 20 videos
worth of full transcripts in one call.
- User uploads a private YouTube video and the app silently
fails — the URL fetch fails because the video is private and
the user did not paste a cookie / OAuth token. Surface the
failure with a specific error: "this YouTube video is private;
download a copy and upload the file instead".
### Negative constraints (hard rules)
- Do NOT invent content not in the lecture. Every bullet, every
card, every concept mention must be backed by a verbatim quote
with a timestamp. A bullet without a quote does not appear.
- Do NOT smooth the lecturer's speech in the verbatim quotes.
"Um", "uh", false starts, mid-sentence corrections — preserve
them when they are load-bearing (a correction is data; an "um"
in the middle of a definition is data). When `is_partial_or_inaudible`
is true, mark the gap as `[inaudible]`, do not fill it in.
- Do NOT translate proper nouns. Lecturer names, institution names,
technical terms with no equivalent (Wirtschaftswunder,
bodhicitta, machismo, jeitinho, han) stay verbatim. Add a
parenthetical gloss only the first time a term appears.
- Do NOT paraphrase formulas. "P(B|A) = P(A|B)P(B)/P(A)" is captured
verbatim, not as "Bayes' theorem".
- Do NOT silently drop slide content. Every load-bearing slide is
in `slides[]` with its `slide_text_verbatim` exact.
- Do NOT bypass YouTube paywalls or download private videos
without the uploader's consent. Public, unlisted-with-user-supplied-URL,
and user-uploaded files only.
- Do NOT use the user's recordings to train or fine-tune any model.
Use the Gemini API on the paid tier, where Google does not use
your content for model training, per the Gemini API Additional
Terms. The capabilities-info panel says this in plain English.
- Do NOT auto-publish or auto-share a course. Sharing is explicit,
per-course, per-classmate, via magic link.
- Do NOT auto-claim that an exam will cover a specific topic. The
app surfaces "the lecturer said this would be on the exam" with
the verbatim quote; the user makes the inference about their own
exam.
- Do NOT generate cards that the lecturer did not implicitly or
explicitly justify. A card asking "what is the molecular weight
of haemoglobin?" is invalid if the lecturer never gave a number;
the card must be cut.
### Per-call `systemInstruction` strings
Use these as the literal `systemInstruction` field for each Gemini API call the built app makes. They complement the series-wide rules already uploaded as the global instructions file (`00-series-instructions.txt`).
### Call: Watch lecture video, parse → `LectureNotes` schema
Model: `gemini-3.5-flash` · thinkingLevel: medium · Tools: (none)
```
You are watching a recorded lecture video. The user is a learner who
will use your structured notes to study for an exam or to consolidate
their understanding of the material. Your job is to be a faithful
reader of the lecture, not a co-author.
The video is uploaded via the Gemini Files API and provided as a
single video input. Watch the entire video before producing output.
Do not output partial notes from the first segment; you have
long-context capacity, use it.
Languages you may encounter: English (American, British, Indian,
Singaporean, Filipino, Nigerian variants), Spanish (peninsular,
Mexican, Argentinian, Andean), Portuguese (Brazilian and European),
Mandarin, Cantonese, Korean, Japanese, Vietnamese, Tagalog (with
Taglish), Tamil (with Tanglish), Hindi (with Hinglish), Urdu,
Bengali, Punjabi (Gurmukhi or Shahmukhi), Amharic, Swahili, Farsi
(with English code-mixing in technical fields), Khmer, Indonesian,
Malay, Thai, Turkish, Polish, Russian, German (Hochdeutsch and
Austrian variants), French (metropolitan, Quebecois, Senegalese,
Maghrebi), Italian, Greek, Hebrew, Arabic (MSA and major regional
dialects). If the lecturer code-switches mid-sentence (common in
Indian English engineering lectures, Hong Kong economics
lectures, Filipino medical lectures, Quebecois philosophy
lectures), preserve the switch in the verbatim quote and note the
languages in source_language as the primary language with a
comment in audio_quality_note.
Detect speakers. If a second person speaks for more than 30
consecutive seconds, treat them as a distinct speaker with their
own SpeakerAttribution entry. Attribution basis options are
named in the schema; pick the strongest you have evidence for.
Output ONLY the LectureNotes JSON matching the provided schema.
Hard rules:
- Every OutlineEntry.bullets[i] must include a supporting_quote.
A bullet without a verbatim quote and a timestamp is invalid.
If you cannot find a verbatim quote for a point, do not include
the point.
- The timestamp on a quote is the second the lecturer BEGAN
saying that sentence, not the second the related slide appeared
or the second the next speaker started. Be exact.
- transcript-style smoothing is forbidden. If the lecturer says
"um, the — the Bayes rule is, uh, P(B|A) equals P(A|B)P(B)
over P(A)", the verbatim quote preserves the disfluencies. If
you must drop disfluencies for the bullet itself, the verbatim
quote still preserves them.
- Slide content is captured separately from speech in slides[].
Every slide that contains substantive text or a formula is
captured. Chrome (institution logo, footer, page number) is
not.
- Formulas on slides are captured verbatim AND as LaTeX in
latex_strings[]. "P(B|A) = P(A|B)P(B)/P(A)" appears in
slide_text_verbatim and as `\Pr(B \mid A) = \frac{\Pr(A \mid B)\Pr(B)}{\Pr(A)}`
in latex_strings.
- Diagrams on slides or whiteboards are described in one sentence,
concrete and sensory. "A line graph showing membrane voltage
rising sharply from -70 mV to +40 mV between t=1ms and t=2ms,
then falling more slowly." Not "an illustrative figure".
- Rewatch windows: pick 3-7 short windows that together total no
more than 8 minutes. Reason from the enum. Each window must
carry a one_line_justification grounded in a verbatim quote.
- Flashcards: typically 30-90 per ninety-minute lecture. Each
card has a supporting_quote and a related_outline_section.
Difficulty maps to Bloom: recall (definitions), understand
(interpretations), apply (worked example variants),
synthesise (cross-concept connections).
- If the audio is poor for a passage (auditorium noise, mic
issues, audience questions away from the mic), set
is_partial_or_inaudible: true on the quote and note in
inaudible_note. Do NOT guess at what was said.
- is_exam_signalled is true only when the lecturer explicitly
flags exam relevance ("this will be on the exam", "you must
know this for the test", "expect a question like this"). Do
not infer from emphasis alone.
- overall_reading_confidence is your honest judgement of how well
you parsed the video as a whole. flagged_for_user_review names
any field_path you are less than 0.7 confident about.
No commentary. JSON only.
```
---
### Call: Translate transcript + outline to target language
Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: (none)
```
You translate a lecture outline and its verbatim quotes from a
source language to a target language the user specifies. The
translation will be read by a learner who studies in the target
language but whose lecturer spoke in the source language.
Hard rules:
- Do NOT translate proper nouns. Lecturer names, institution
names, theorem names (Hodgkin-Huxley, Black-Scholes,
Gödel-Bernays), technical terms with no equivalent. Add a
parenthetical gloss the first time only.
- Do NOT translate formulas. P(B|A) = P(A|B)P(B)/P(A) is the same
in every language.
- Preserve the structure exactly. Same outline depth, same
number of bullets per section, same flashcard count. Translate
field values; do not restructure.
- Verbatim quotes are translated faithfully. Disfluencies and
false starts are translated as faithfully as possible ("um" →
"ahm" in German, "э-э" in Russian, "あの" in Japanese). When a
disfluency has no equivalent, drop it but note in
flagged_for_user_review.
- Technical terminology: when the target language has a standard
technical term, use it (German "Aktionspotential" for "action
potential"). When it does not, keep the English term in italics
and add a one-line gloss.
- Code-switching in the source is preserved in the target. If a
Filipino medical lecturer says "ang ECG natin shows a STEMI" in
Tagalog with English medical terms, the English translation is
"our ECG shows a STEMI" with a note that the original mixed
Tagalog and English clinical vocabulary.
Output: the translated LectureNotes JSON, structurally identical
to the source, with field values translated and source_language
left unchanged (because the source language is a property of the
original recording, not the translated notes). No commentary.
```
---
### Call: Generate Anki flashcards from `LectureNotes`
Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: (none)
```
You take a parsed LectureNotes object and generate a deck of
spaced-repetition flashcards. The user will export the cards to
Anki, Quizlet, or RemNote.
Hard rules:
- Every card has a supporting_quote field populated with a
verbatim quote from the lecturer and a timestamp. A card
without a verbatim quote is invalid and must not be generated.
- Cards are short. Front ≤ 140 characters. Back ≤ 280 characters
unless back_is_verbatim_quote is true (in which case the back
is the lecturer's exact words and may be longer).
- Definitions become recall cards. "What is the resting membrane
potential of a typical neuron?" Back: "-70 mV". Supporting
quote: "the resting potential of a typical neuron is around
minus seventy millivolts" at 00:14:22.
- Worked examples become apply cards. The front sets up the
problem; the back is the lecturer's worked answer, verbatim
where possible.
- Cross-concept connections become synthesise cards. The front
asks the learner to relate two ideas the lecturer linked; the
back cites both lecturer quotes.
- Do NOT generate cards that test content not in the lecture. If
the lecturer did not state a fact, you cannot ask the learner
to recall it.
- Do NOT generate cards that paraphrase the lecturer in a way
that changes meaning. If the lecturer said "approximately -70
mV", the back of the card says "approximately -70 mV", not
"exactly -70 mV".
- Tag cards with related_concept_tags so the learner can filter
to a topic later.
- Aim for 30-90 cards per ninety-minute lecture. Quality over
quantity. A lecture that only justifies 22 cards should
produce 22, not 60.
Output: the Flashcard[] array as JSON. No commentary.
```
---
### Call: Rank re-watch windows from `LectureNotes`
Model: `gemini-3.5-flash` · thinkingLevel: medium · Tools: (none)
```
You take a parsed LectureNotes object and produce the canonical
"seven-minute re-watch list" — the short ranked list of windows
the learner should actually re-watch before their exam.
Hard rules:
- Total combined duration across all returned windows is at most
8 minutes (480 seconds). If you cannot fit the most important
moments in 8 minutes, return the top-priority subset and note
in flagged_for_user_review.
- Each window is 30-180 seconds. Windows shorter than 30 seconds
do not give the learner enough context; windows longer than
180 seconds defeat the point.
- Pick from the enum reasons. Prefer:
1. exam_signalled (the lecturer said this is on the exam)
2. key_definition_introduced (a term the rest of the lecture
builds on)
3. worked_example (a worked problem variant of which appears
on exams)
4. slide_too_dense_for_realtime (a slide with five formulas the
lecturer spoke past in 20 seconds)
5. concept_that_recurs_later (foreshadowing for a later week)
6. common_misconception_addressed (the lecturer corrected a
widespread misunderstanding)
7. lecturer_slowed_down (a deliberate emphasis cue)
8. user_pinned (always rank=0, always included regardless of
budget; if user_pinned exists, trim the others to fit)
- Each window has a one_line_justification grounded in the
supporting_quote. The user must be able to read the
justification and the quote, and immediately understand why
this window is on the list.
- Rank 0 is the most important. Do not invent ties; if two
windows are equally important, pick one as 0 and one as 1
based on chronological order in the lecture.
- Do NOT include windows that are merely "interesting". The user
has a 9am exam; the bar is "would not pass without this".
Output: the RewatchWindow[] array as JSON. No commentary.
```
---
### Call: Course-wide cross-lecture concept linking
Model: `gemini-3.5-flash` · thinkingLevel: medium · Tools: (none) — long-context over the course
```
You receive the parsed LectureNotes of every video in a course at
once (long-context). Your task: identify concepts that recur
across multiple lectures, and produce a graph that links them.
Hard rules:
- A "concept" is a named idea the lecturer treats as a unit
(Hodgkin-Huxley model, Bayes' theorem, eigenvalue decomposition,
Krebs cycle, Polish vowel reduction, Black-Scholes equation,
hypothesis-testing framework, etc.). It is not a single word.
- Two mentions are the "same concept" only with strong textual
evidence: identical or near-identical naming, identical or
near-identical formula presentation, or explicit back-reference
by the lecturer ("recall from week 3").
- Output a graph: nodes[] (each concept), edges[] (each
cross-reference between mentions in different videos), with
confidence levels.
- Identify the canonical introduction (the first lecture and
timestamp where the concept was introduced) and every revisit
(subsequent mentions, ranked by depth of treatment).
- The user is the final arbiter. Surface low-confidence edges
with a flag.
Output: the concept graph JSON as nodes[] + edges[]. No commentary.
```
---
### Call: "Did the lecturer say X?" strict-quote answer
Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: (none)
```
You receive a parsed LectureNotes object (or several, for a
course-wide query) and a user question of the form "did the
lecturer say anything about X?". Your job is to answer yes, no,
or no-clear-answer, with the supporting quote and timestamp if
the answer is yes.
Hard rules:
- The answer is grounded ONLY in the LectureNotes provided. You
do not have access to the open web for this call.
- If the lecturer addressed X explicitly, return:
{ answer: "yes", supporting_quote: VerbatimQuote }
- If the lecturer did not address X at all in the provided
material, return:
{ answer: "no", supporting_quote: null,
note: "the lecturer did not discuss X in the provided
material" }
- If the lecturer touched on X tangentially or X is implied but
not stated, return:
{ answer: "no-clear-answer",
closest_supporting_quote: VerbatimQuote,
note: "the lecturer mentioned [closest related concept] but
did not directly address X" }
- Do NOT extrapolate from background knowledge. If you "know"
the answer from elsewhere, that is irrelevant. The user is
asking whether the lecturer said it.
- Quote verbatim. Do not summarise the quote.
Output: the structured answer as JSON. No commentary.
```
---
### Call: Generate TTS audio summary in source/target language
Model: `gemini-3.1-flash-tts-preview` · n/a · n/a
```
Voice: clear, unhurried. Pick the Gemini 2.5 Flash TTS voice
whose `languageCode` matches the target_language the user has
chosen for audio playback — pronunciation will follow that
locale automatically. Prefer the gender published for the locale
that the user has not heard most recently (rotate when the user
has multiple commute summaries). Fall back to whichever voice
is available rather than blocking.
Pre-process the outline text before sending it to TTS:
- Read the section_title, then a brief pause (single ellipsis
"…"), then each bullet's `point` field.
- At each section break, insert a blank line plus an em-dash
("—"). Gemini 2.5 TTS does not support SSML `` —
these textual cues are how you signal pace.
- Skip timestamps and quote citations from the spoken summary
(they are visual metadata, not spoken content). The on-screen
subtitle shows the timestamp; the audio does not.
- Skip formulas when the target_language has no clean spoken
form for them. Render formula references as "see slide" with
a subtitle pointing at the formula.
- Mid-call voice switching is not supported. If the source
lecture is bilingual (Tagalog + English clinical terms), pick
one TTS voice and accept that the proper-noun pronunciation
will be best-effort. Optional: stitch a second TTS call
client-side for emphasised foreign terms.
- Target rate: ~140 words per minute — commute pace, slightly
faster than letter-reading pace from Letters in the Attic.
Style direction: prepend ONE short directive sentence to the
text input, exactly like: "Read clearly and unhurriedly, as a
study companion on a commute. …". There is no separate `style`
API field on Gemini 2.5 TTS; the directive sentence inside the
input is how style is conveyed.
Phoneme overrides (Polish ł, German ü, Hebrew final letters,
Arabic emphatic consonants, tonal markings in Mandarin,
Vietnamese tones) are NOT exposed by Gemini 2.5 TTS — no SSML
`` tag. Pronunciation comes from the chosen voice's
native locale.
```
---
### Call: Generate slide-frame crop description
Model: `gemini-3.5-flash` · thinkingLevel: low · Tools: (none)
```
You receive a cropped image of one slide from a lecture video.
The slide contains a diagram, figure, or visual element that
needs a sentence-level description for the outline notes.
Describe the slide in one sentence — concrete, specific,
sensory. Avoid art-history vocabulary. Avoid speculation about
the lecturer's intent. The description goes into
slides[].diagram_description and will be shown to the learner
who is reading the notes without playing the video.
Examples of good descriptions:
- "A line graph showing membrane voltage rising sharply from
-70 mV to +40 mV between t=1ms and t=2ms, then falling more
slowly to -80 mV by t=4ms."
- "A flow diagram of the Krebs cycle with eight labelled
intermediates arranged in a clockwise loop, with three arrows
marked 'CO2 released'."
- "A whiteboard sketch of a normal distribution with the mean
labelled μ, the standard deviation shaded between -σ and +σ,
and the area outside shaded a darker tone."
Examples of bad descriptions:
- "An illustrative figure" (too vague)
- "A masterful exposition of the underlying physiology" (too
rhetorical)
- "The lecturer clearly wanted to emphasise the action potential"
(interpretation, not description)
Output: a single string. No commentary.
```
## 5. Use cases & content to include
Build dedicated UI sections or flows for each of these — they tell you what content the app must support.
- **The shelf-exam eve.** Priya, a second-year medical student in
Boston with a 9am cardiology shelf exam, drops in the URLs of nine
ninety-minute lectures she half-attended over the semester. By
midnight she has a deck of 540 cards, an outline she can search,
and a seven-minute re-watch list that includes a thirty-second
window from week 4 where the cardiologist said "this exact ECG
pattern will appear on the exam".
- **The qualifying exam.** Mariana, a PhD candidate in Lisbon, has
twelve hours of Brazilian Portuguese econometrics lectures from
her supervisor's department channel to revise before her qualifying
exam. She runs them through the app in source-language mode
(transcripts in Portuguese, outline in Portuguese, cards in
Portuguese). The app distinguishes the supervisor's voice from the
postdoc who introduced two of the recorded sessions.
- **The Korean-grad-student conference catch-up.** Jisoo, a grad
student in Seattle, missed the first two days of a six-day
conference and the talks are now on YouTube. She drops in fourteen
recordings, asks the app to identify the talks that referred back
to "her" research area, and gets a course-wide cross-lecture concept
map showing how three speakers built on a single 2023 paper.
- **The non-English lecture series.** A learner in Manila is working
through a forty-hour University of the Philippines Diliman lecture
series in Tagalog on Philippine constitutional law. The app
transcribes in Tagalog, generates the outline in Tagalog, and
produces Tagalog flashcards. The user also wants an English
side-by-side for her American husband who is curious about the
course; the second translation pass runs overnight.
- **The bootcamp recap.** A bootcamp learner in Lagos has been working
through Andrej Karpathy's "Zero to Hero" YouTube series. After
finishing video 4 of 10, she runs the whole series through the app
and discovers a fifth-video moment where Karpathy says "you will
thank me later for this" — she had skimmed past it.
- **The multi-speaker journal club.** A radiology fellow records her
weekly journal club and uploads the file. The app distinguishes the
attending who chaired, the fellow who presented, the resident who
asked the question that broke the discussion open. The cards are
tagged by speaker so the fellow can revise her own contributions
separately.
- **The bad audio.** A grad student records the lecture on her phone
from the back of a 400-seat auditorium. The audio is poor in the
back half of the recording. The app processes what it can and
flags four passages as "low confidence — listen yourself"; the
user knows exactly which seven minutes she has to re-listen to
herself, not which ninety she has to re-watch.
- **The CME module.** A nurse practitioner in rural Ontario has six
hours of mandatory continuing-medical-education modules due before
Friday. She drops them in, gets the outline and cards, and the
app pre-fills a "I have completed and reviewed" form for the
licensing body — citing the timestamps she watched.
- **The Bach masterclass.** A retired engineer in Hamburg is working
through twenty hours of recorded harpsichord masterclasses on a
YouTube channel. The lectures are in German with occasional French
interjections from the visiting maestro. The app handles the
code-switching and produces an outline keyed to specific bars of
the Goldbergs (the lecturer says "look at bar 47 of the aria";
the outline records that and links to the bar reference).
## 6. Page structure
Build the following screens / sections in this order. Adjust copy to fit the voice, but keep the structural intent.
1. **Welcome / sign-in.** A photographed-looking image of a desk at
11pm: a tablet propped up showing a lecture playing at 2x, a
notebook with handwritten notes, a cold mug, the corner of a
textbook. One paragraph: "Lecture, Condensed turns hours of
lecture video into structured notes, flashcards, and the seven
minutes you actually need to re-watch — every claim timestamped,
every quote verbatim." Single Google sign-in button; Apple
sign-in next to it. Below: "Try with the sample course" → loads
the demo course in section 8a.
2. **Empty state — "Start a course".** Three input methods: 🔗 Paste
YouTube / Vimeo URLs (one per line) · 📁 Upload video files · 📚
Import from a Panopto / Kaltura course. A short explainer below
each ("Best for public or unlisted lectures", "Best for
self-recorded or paid-course videos", "Best when your institution
uses Panopto and you have access").
3. **Course setup.** The user names the course ("Cardiology Block 3"),
picks the target language for notes (defaults to UI language),
confirms the source language (defaults to auto-detect), and
chooses whether to include flashcards (default yes), TTS audio
summaries (default no), and a course-wide concept map (default
yes if more than three videos).
4. **Processing queue.** A vertical list of every video in the course.
Each item shows the video thumbnail, duration, and a step-by-step
honest progress bar: "Watching the lecture…" → "Building the
outline…" → "Picking re-watch windows…" → "Drafting flashcards…".
Each video takes 2-6 minutes for a ninety-minute lecture on
Gemini 3.5 Flash. The user can close the app and come back; results
stream in as each video completes.
5. **Lecture detail view.** A three-column layout on desktop, stacked
on mobile. Left column: the embedded video player with the
outline-section markers on the timeline (so the user can jump to
any section by clicking the marker). Middle column: the outline,
collapsible by section, with each bullet showing the lecturer's
verbatim quote in a smaller, lighter font directly underneath the
bullet point. Right column: the seven-minute re-watch list with
one-tap "play this window" controls that auto-play the video at
the start time and pause at the end time. Sticky header: lecture
topic → duration → primary speaker → "(i) show how the AI read
this".
6. **Outline navigation.** A keyboard-driven outline view where the
user can press `j` / `k` to move between bullets, `Enter` to jump
the video to the bullet's timestamp, `space` to add a bullet to a
"make a card from this" queue, and `r` to mark a bullet's window
for re-watch (forcing user_pinned).
7. **Re-watch player.** A full-screen player that auto-plays each
re-watch window in turn, pauses, and surfaces a one-tap "watched
— next" button. The bottom of the screen shows the total elapsed
re-watch time so the user can see they are six minutes into their
seven.
8. **Flashcard study mode.** A standard spaced-repetition study UI:
front of card → reveal back → easy / good / hard / again. Each
card's back shows the lecturer's verbatim quote in a quote-block,
with a small "▶ play this moment" button that opens the video at
the timestamp. The cards can be filtered by concept tag, by
lecture, by speaker, or by difficulty.
9. **Quiz mode.** A 10-minute self-test drawn from the active
lecture's cards. Multiple-choice for recall, free-text for apply,
compare/contrast for synthesise. After scoring, the app produces
a focused re-watch list of the windows behind the cards the user
missed.
10. **Course-wide concept map.** A graph view where each node is a
concept that appears in more than one lecture, and each edge is
a back-reference. Clicking a node lists every lecture and
timestamp where the concept appears. The graph is keyboard-
accessible (arrow-key navigation between nodes; list-mode
alternative).
11. **Did-the-lecturer-say question box.** A persistent input near
the top of the lecture detail view: "ask whether the lecturer
said anything about __". Answers are yes / no / no-clear-answer
with a verbatim quote and timestamp deep-link.
12. **Sharing.** Modal: "Share this course read-only with a
classmate". Magic-link email; arrival drops the classmate
straight into the course as a viewer. Classmate can see notes,
cards, and re-watch lists but the source videos open in their
native platform (YouTube, Vimeo, the institution's Panopto).
13. **Export.** PDF study handout, Anki `.apkg` deck, Quizlet `.csv`,
RemNote / Obsidian markdown with `[[hh:mm:ss]]` deep-links to
the source video.
14. **Footer.** "Made for the night before the exam." Privacy: "Your
recordings are yours. We never train on them." Capabilities
`(i)` icon in header.
## 6b. First-visit onboarding
Show a **first-visit onboarding** the first time a visitor lands on the app (detect via `localStorage` flag; do not show on return visits). Three slides, dismissible at any time. Persistent re-entry: a `?` icon in the header reopens it.
**Slide 1 — What this is.**
- Headline: "Welcome to Lecture, Condensed."
- Subhead: "Turn hours of lecture video into the seven minutes you actually need to re-watch — in any subject, any language, with every claim timestamped."
- One paragraph (≤ 60 words) explaining who this is for and what makes it different from a generic transcription tool: it cites the lecturer's verbatim words behind every claim, it identifies the windows that are actually worth re-watching, and it never invents content the lecturer did not say.
- Visual: a small annotated illustration of a lecture-detail view with the relevant elements labelled (timestamped bullet, verbatim quote under it, re-watch window on the right) — not a generic play-button icon.
**Slide 2 — Try it now.**
- One short prompt: "Try with the sample course".
- A live demo input pre-loaded with three short lectures from the seed content in section 8a (one English, one Brazilian Portuguese, one Korean).
- 1-2 sentences pointing at *the specific page elements* where the Gemini magic happens (the side-by-side source + translation in the Portuguese lecture, the seven-minute re-watch list in the English lecture, the speaker-attribution panel in the Korean panel-style session).
**Slide 3 — How to remix this.**
- Headline: "Make this yours."
- Three short bullets:
- "Swap the sample course in `/data/seed-course/` for your own videos."
- "Adjust the prompts in `/server/prompts/` to fit your discipline's terminology."
- "Wire up your Gemini API key and Firebase project via the env-var list in the capabilities panel."
- Primary CTA: "Use this template" → links to AI Studio Build remix entry point.
- Secondary: "Just exploring — close" (sets localStorage flag, never auto-shows again).
**Accessibility:** focus trap, `Esc` closes, `role="dialog"`, `aria-modal="true"`, `aria-labelledby`, focus restored to trigger on close. Respect `prefers-reduced-motion`.
**Don't:**
- Don't gate content behind the modal. The page beneath must be fully usable.
- Don't auto-reshow on return visits. Use `localStorage['onboarding-seen-v1']`.
- Don't include unrelated CTAs (newsletter signup, social follow). Keep it about the template only.
## 6c. Capabilities info button (persistent in header)
Add a persistent `(i)` icon in the top-right of the header (next to the primary nav). Click → opens a modal/panel titled **"What powers this app"**.
**Panel contents (in this order):**
**Gemini capabilities used (the hero list):**
- **Gemini 3.5 Flash (long-context video)** — the hero capability. Watches
the full lecture in one call: speech, slide content, on-screen text,
whiteboard writing, laser-pointer emphasis, the moment the lecturer
says "this is exam-relevant". One call per video, not per chunk.
- **Gemini 3.5 Flash (multilingual)** — transcribes and translates in
Tagalog, Vietnamese, Mandarin, Cantonese, Korean, Tamil, Hindi,
Urdu, Bengali, Punjabi, Amharic, Swahili, Farsi, Khmer, Portuguese,
Spanish, French, German, Russian, Polish, Turkish, Indonesian, Thai
— with technical-terminology preservation.
- **Gemini 3.5 Flash (long context)** — once the course grows, the
cross-lecture concept-linking call sees every video at once to
identify recurring concepts.
- **Gemini 3.5 Flash** — produces slide-frame descriptions and supports
the lower-cost translation pass on demand.
- **Gemini TTS** — optional audio summaries for commute-mode learning,
in the user's chosen language at commute pace.
- **Firebase Auth** — Google and Apple sign-in, course-sharing via
magic links.
- **Firestore** — stores your course, syncs across devices in real
time.
- **Firebase Storage** — keeps your uploaded video files private to
your account (when the lecture is not on YouTube).
- **Cost note** — see the detailed breakdown in 6d. A typical
ninety-minute lecture costs about $0.40 of Gemini API spend; a
semester course of 12 lectures costs about $5.
- **Privacy note** — your recordings, transcripts, notes, and
flashcards are private to your account and the classmates you
explicitly share with. This app uses the Gemini API on the paid
tier, where Google does not use your content for model training,
per the Gemini API Additional Terms. Unlisted YouTube URLs are
treated as confidential.
**Backend services this app depends on:**
- Auth: see section 4b
- Database: see section 4b
- Storage: see section 4b — Firebase Storage is required and is NOT
auto-provisioned; enable it in the Firebase console before first
video upload
- Email: see section 4b — magic-link email requires the sender
domain to be authorised in Firebase Auth
- Payments: see section 4b (not used in v1)
- External APIs: see section 4b — Gemini API for all intelligence;
YouTube Data API for public-video metadata only
**Environment variables you'll need to configure:**
- `GEMINI_API_KEY` — your Google AI Studio API key
- `FIREBASE_PROJECT_ID` — your Firebase project id
- `FIREBASE_SERVICE_ACCOUNT` — service-account JSON (server-side only)
- `YOUTUBE_DATA_API_KEY` — optional, only if you want to fetch
YouTube metadata server-side
**Cost + privacy notes:**
- One short paragraph per cost-sensitive capability: long-context
video calls are billed per token of input — a ninety-minute lecture
parsed at Gemini 3.5 Flash medium-thinking costs about $0.35-$0.45 per
video; the course-wide concept-linking call on a 12-video course
costs about $0.20 per run (default: once at course creation, then
on demand).
- One short paragraph on privacy: where the data lives (your Firebase
project), how to delete it (Settings → "Delete this course forever"
— gone in 60 seconds), what is never sent for training. Uploaded
video files in Firebase Storage are retained only as long as the
user keeps the course; deleting the course deletes the files.
**Documentation links:**
- AI Studio Build docs
- Gemini API multimodal, long-context, video-input, TTS docs
- Firebase Auth, Firestore, Firebase Storage docs
- Anki `.apkg` field-order reference (for the deck export)
**Accessibility:** same standards as the onboarding modal — focus
trap, `Esc`, ARIA, restored focus.
**Behaviour:**
- Always available — single click from anywhere in the app.
- Tooltip on the `(i)` icon: "How this app is built".
- Mobile: opens as a full-screen sheet that slides up.
- Should be the most honest part of the app — never hand-wave
service requirements; never say "AI" without naming the specific
Gemini model and capability.
## 6d. Detailed cost breakdown (deployer reads this BEFORE shipping)
- **Watch lecture video, parse → `LectureNotes`** (Gemini 3.5 Flash,
medium thinking) — a ninety-minute lecture at 1 fps video sampling
≈ ~25-35k input tokens (video) + ~3k output tokens. At 3 Pro input
$1.50/M and output $9/M, that is roughly $0.04 input + $0.015
output ≈ $0.05-$0.07 per ninety-minute video at low-medium
fidelity. At higher-fidelity sampling (2-4 fps for slide-heavy
lectures) the input balloons to 80-150k tokens ≈ $0.10-$0.20 per
video. Budget $0.20 per video as a safe upper bound.
- **Translate transcript + outline** (Gemini 3.5 Flash, low thinking) —
typical 3-4k token notes object both directions ≈ ~$0.02-$0.03 per
lecture per target language.
- **Generate Anki flashcards** (Gemini 3.5 Flash, low thinking) — input
is the `LectureNotes` (~3k tokens), output is the card array
(~5-8k tokens for a 60-card deck) ≈ ~$0.04 per lecture.
- **Rank re-watch windows** (Gemini 3.5 Flash, medium thinking) — input
is the `LectureNotes`, output is a small array (~1k tokens) ≈
~$0.01 per lecture.
- **Course-wide cross-lecture concept linking** (Gemini 3.5 Flash,
medium thinking, long-context) — input is the merged
`LectureNotes` from every video in the course. A 12-video course
≈ ~40k tokens input. Output is the concept graph (~5k tokens).
≈ ~$0.08 per run. Re-runs only when the user adds a video to the
course.
- **"Did the lecturer say X?"** (Gemini 3.5 Flash, low thinking) — input
is the `LectureNotes` for the lecture(s) in scope, output is a
small structured answer (~300 tokens). ≈ ~$0.005-$0.01 per
question.
- **TTS audio summary** (Gemini 2.5 Flash TTS) — billed per output
token (~$10/M output tokens), effectively ~$0.000003/character. A
10-minute summary of a ninety-minute lecture ≈ ~10k characters ≈
$0.03 per summary. Cached per lecture per language.
- **Slide-frame description** (Gemini 3.5 Flash, low thinking) — ~$0.001
per slide. A lecture with 30 slides ≈ $0.03 total.
- **Expected per-lecture cost on first ingest:** ~$0.30-$0.45.
**Semester course of 12 lectures total:** ~$4-$6. **Ongoing
question-answering** (10 questions per week per course): ~$0.10/week.
- **Video storage:** Firebase Storage standard tier, ~$0.026/GB/month.
A 90-minute uploaded lecture at 720p ≈ 600 MB; a 12-lecture course
uses ~7 GB ≈ ~$0.18/month. Encourage users to use YouTube/Vimeo
links rather than file upload when possible — there is no storage
cost on linked videos.
## 7. Design language
- **Mood:** The desk at 11pm before the exam. A tablet playing a
lecture at 1.5x in the corner of the eye, a notebook half-full,
the discipline of a study tool that respects the user's time. Not
a flashy ed-tech product. Not a productivity app with a streak
counter. The quiet competence of a really good study guide
printed on better paper than the rest of your stack.
- **Typography:** Display serif for the outline body and lecture
topic headings (Source Serif Pro or Adobe Caslon Pro). A clean
grotesque for the chrome and the timestamps (Inter or Geist). A
monospace for formula rendering and for the raw `hh:mm:ss`
display next to bullets (JetBrains Mono or IBM Plex Mono).
- **Palette:** Off-white study-paper background `#FAF7F2` for the
outline view, deep ink `#15171B` for body text, calm blue `#2D4A6E`
for timestamps and section markers, warm amber `#B5651D` for
exam-signalled highlights and the seven-minute re-watch badge, a
muted green `#3E6B47` only for "correct" / "watched" confirmations,
a restrained red `#9A3232` only for "low-confidence" flags and
"could not read" notices. Borrowed from a well-designed printed
study guide, not from an ed-tech SaaS.
- **Imagery:** The video player is the hero in the lecture detail
view; never decorated with frames, badges, or chrome. Slide-frame
crops appear at their native aspect ratio with a one-pixel
off-white border so they read as captured artefacts. Speaker
thumbnails (where the model identifies a face) are small,
greyscale, and only on multi-speaker panels.
- **Hand-feel touches:** A barely-visible paper grain on the
outline background. The seven-minute re-watch list has a thin
warm-amber spine on the left side, suggestive of a bookmark
ribbon. The "play this window" button is a flat triangle, not
a YouTube-style red circle.
- **Spacing:** consistent 4-px base. Generous whitespace — the
outline needs air to be scannable.
- **Radius:** consistent token set (e.g. 6 / 12 / 20 px). Outline
cards use 6; the video player container uses 12; the welcome card
uses 20.
- **Shadows:** subtle, layered, neutral-warm. Avoid heavy
drop-shadows. The re-watch list lifts off the page with a soft
shadow at hover, never aggressively.
- **Motion:** purposeful — entrance fades, hover lifts, timeline
marker pulses (briefly, once) when a section is jumped to.
Respect `prefers-reduced-motion`. No bouncing splash animations.
No theatrical hero animations. The seven-minute re-watch player's
auto-advance between windows is the one place where motion
carries meaning; respect reduced-motion by jumping rather than
cross-fading.
- **States:** every interactive element has hover, focus, active,
disabled. Loading uses skeletons not spinners where possible.
Empty states have helpful next-action guidance ("Paste a YouTube
URL or upload a lecture file to start").
## 8. Content generation rules
- Write **realistic, specific copy**. NO Lorem Ipsum. NO generic
placeholders like 'Your tagline here'.
- Invent plausible lecturer names, video titles, course topics,
timestamps, quotes, and slide content that fit the domain (use the
seed content in section 8a as a starting point). When inventing,
lean on real disciplines but never claim a fictional lecture is a
real recording by a real named professor. Use "Dr A. Martins" or
"Professor Lee", not "Anthony Fauci".
- Tone: warm, direct, free of corporate language. This template is
for a person studying for an exam, not for a company selling
productivity.
- Headlines: punchy and concrete. No 'Empower your X' filler. No
'Revolutionize'. No 'Seamless'.
- Body copy: short paragraphs (2-4 sentences). Use lists where
appropriate.
- Plain language. Avoid jargon — except where the user already
speaks the jargon (the medical student wants to see "STEMI" in the
cardiology demo; the econometrics PhD wants to see "GMM"; the
Korean grad student wants to see "표본추출" in her notes).
- Where the app outputs AI-generated content, never label it as "AI
says" — let it speak naturally. Use small uncertainty cues only
where epistemic honesty requires them (a low-confidence bullet
shows a faint amber tick; tapping it reveals the alternates the
model considered).
## 8a. Seed content (use these specific examples)
Anchor every generated copy + sample data point in the concrete content below. Use these names, numbers, dates, and snippets verbatim where helpful, or generate close variants that sit in the same world.
**Sample courses (sidebar):**
- "Cardiology Block 3 — Action Potentials" (9 lectures, just me) —
English, second-year medical school recordings from a US
east-coast university, ninety-minute lectures, primary lecturer Dr
M. Roy, occasional guest lectures from a cardiac electrophysiology
fellow. 12.5 hours total. Includes one panel discussion with three
panellists.
- "Econometria II — Métodos Quantitativos" (8 lectures, contributors:
me, two classmates) — Brazilian Portuguese, PhD-level lecture
series from an unnamed Lisbon department, lecturer Dr A. Martins,
occasional guest lecturer Prof. R. Sá from the postdoc cohort.
11 hours total.
- "Machine Learning — 표본추출과 추론" (6 lectures, just me) — Korean
with English technical terms, MS-level course at a Seoul
university, lecturer Professor Lee, includes a journal-club style
multi-speaker session in week 4.
- "Andrej Karpathy — Zero to Hero" (10 public YouTube lectures,
shared with three classmates) — English, primary lecturer Andrej
Karpathy, used for a self-organised study group in Lagos.
**Sample lecture in detail view (this is what the demo should show):**
- **Course:** Cardiology Block 3 — Action Potentials
- **Lecture title:** "Lecture 4 — The Hodgkin-Huxley Model and ECG
Correlates"
- **Source URL:** unlisted YouTube link (from the university's
Panopto-mirrored channel)
- **Duration:** 01:32:14
- **Source language:** en-US
- **Primary speaker:** Dr M. Roy (detected via title card at 00:00:08
and voice consistency throughout)
- **Other speakers detected:** one questioner from audience at 00:47:22
(~22 seconds), unattributed; one guest lecturer Dr P. Singh at
01:08:14-01:22:09 (~14 minutes), introduced by name
- **Topic (one line):** "The Hodgkin-Huxley model of the action
potential, with clinical correlates on the surface ECG"
- **Topic (long form):** "Dr Roy walks through the four ionic
currents underlying the cardiac action potential, derives the
Hodgkin-Huxley equations as adapted for ventricular myocytes,
then maps each phase of the action potential to its corresponding
feature on the twelve-lead ECG. The final twenty minutes,
delivered by Dr Singh, focuses on three exam-classic ECG patterns
(anterior STEMI, hyperkalaemic T-waves, long-QT syndrome) and
links each back to the underlying ionic disturbance."
- **Sample outline entry:**
- section_title: "Phase 0 — fast sodium current"
- section_window: { start: 00:14:22, end: 00:21:08 }
- bullet 1:
- point: "Phase 0 of the cardiac action potential is the rapid
upstroke driven by voltage-gated sodium channels."
- supporting_quote.text: "phase zero is the fast upstroke; you
see the membrane voltage swing from about minus eighty up to
plus twenty, plus thirty millivolts in well under two
milliseconds, and the whole show is run by the fast sodium
channels"
- supporting_quote.timestamp: 00:14:44
- related_slide_index: 3
- confidence: 0.96
- bullet 2:
- point: "The fast sodium channels inactivate within about a
millisecond, which is why phase 0 is so brief."
- supporting_quote.text: "now these channels are spectacularly
fast — they open within about a millisecond, but they also
inactivate within about a millisecond, which is exactly why
the upstroke is so steep and so short"
- supporting_quote.timestamp: 00:15:36
- related_slide_index: 3
- confidence: 0.94
- is_definition: true
- is_exam_signalled: true (Dr Roy at 00:17:12: "you need to know
the four ionic currents and which phase they govern — that is
the highest-yield item on the cardiology shelf")
- is_worked_example: false
- **Sample slide capture:**
- slide index 3, appeared_at 00:14:22, visible_until 00:21:08
- slide_text_verbatim: "Phase 0 — fast Na+ current\nINa: rapid
upstroke\n-80 mV → +20 mV in <2 ms\nINa inactivates within
~1 ms (h-gate)"
- contains_equation: false
- contains_diagram: true
- diagram_description: "A line graph showing membrane voltage
rising sharply from -80 mV to +20 mV between t=0 ms and t=2 ms,
with the segment labelled 'INa active' and a shaded region
marking sodium-channel inactivation around t=1 ms."
- is_load_bearing: true
- **Sample re-watch windows (top 3 of 5; total 6 min 48 s):**
- window 1 (rank 0): 00:14:22–00:16:30 (2 min 8 s), reason
"key_definition_introduced", one_line_justification "the
foundational definition of phase 0 that the rest of the lecture
refers back to", supporting_quote at 00:14:44
- window 2 (rank 1): 00:17:00–00:18:10 (1 min 10 s), reason
"exam_signalled", one_line_justification "Dr Roy explicitly says
this is the highest-yield item on the cardiology shelf",
supporting_quote at 00:17:12
- window 3 (rank 2): 01:14:30–01:18:00 (3 min 30 s), reason
"worked_example", one_line_justification "Dr Singh works the
anterior STEMI ECG with reference to phase 0 — exam-style case
presentation", supporting_quote at 01:14:48
- **Sample flashcards (3 of 67):**
- card 1, recall: front "What is the resting membrane potential of
a typical ventricular myocyte?" back "about -80 mV"
back_is_verbatim_quote false, supporting_quote.text "the resting
membrane potential of a typical ventricular myocyte sits at
about minus eighty millivolts" at 00:09:12
- card 2, understand: front "Why is phase 0 of the cardiac action
potential so brief?" back "Because the fast sodium channels
inactivate within about a millisecond of opening — the upstroke
is steep and short because the channels open and inactivate on
the same timescale" back_is_verbatim_quote false, supporting_quote
at 00:15:36
- card 3, apply: front "On a twelve-lead ECG, which feature
corresponds to phase 0 of the ventricular action potential?"
back "The QRS complex — specifically the steep R-wave upstroke
in the precordial leads" back_is_verbatim_quote false,
supporting_quote at 00:42:10
- **Reading confidence:** 0.93
- **Audio quality note:** "lecturer near mic throughout; audience
question at 00:47:22 partially inaudible — flagged"
**Sample input artefacts (for the build to demonstrate):**
- A 90-minute English medical-school lecture on the cardiac action
potential from a public university channel.
- A 75-minute Brazilian Portuguese econometrics lecture from a Lisbon
PhD course with technical English terms code-mixed in
("heteroscedasticity", "GMM", "instrumental variables").
- A 60-minute Korean machine-learning lecture from a Seoul university
course with English technical terms ("attention", "logits",
"sampling").
- A 25-minute Andrej Karpathy YouTube video (public) for a less
formal lecture format.
- A 45-minute self-recorded radiology journal club (uploaded file)
with three speakers: the chair, the presenter, and a discussant.
**Sample voice copy:**
- Onboarding: "Drop in your lecture videos. We'll watch them — even
the slow bits — so you can re-watch only what counts."
- Processing: "Watching the lecture…" / "Building the outline…" /
"Picking the re-watch windows…" / "Drafting the flashcards…"
- Empty course: "This course is waiting for its first lecture. Paste
a YouTube URL or upload a video to start."
- Error (unreadable video): "We couldn't process this video. If it's
a private YouTube link, try downloading the file and uploading it
instead."
- Save confirmation: "Added to Cardiology Block 3 — Lecture 4, the
Hodgkin-Huxley Model. 67 cards, 7 minutes of re-watch."
- Exam-signalled highlight: "Dr Roy flagged this as exam-relevant.
This window is in your re-watch list."
- Low confidence note: "Some passages were hard to transcribe.
Tap any flagged passage to see what we heard."
**Sample course-share email subject + body:**
- Subject: "Priya — I built notes for the cardiology block. Want in?"
- Body: "Hi Priya — I ran the nine block-3 lectures through this
thing. Notes + cards + re-watch list are ready. Tap to view (your
notes are private to your account; the course view is read-only
for you)." [Open Course]
## 9. Media & assets
- **Hero image (landing screen):** A photographed-looking shot of a
desk at 11pm: a tablet propped on a textbook playing a lecture at
1.5x with the playhead near the end, a notebook with handwritten
shorthand, a cold mug of tea (not coffee — the user is studying
late, the coffee was earlier), the corner of an iPad with the
outline view. Generate via Nano Banana 2 with a prompt emphasising
"warm desk-lamp light, late evening, hand of a student in their
twenties (rotate between Indian-American, Brazilian, Korean-American
for variation), realistic textbook with notes spilling out of it,
a quiet study atmosphere".
- **App icon / wordmark:** Set in the display serif. A small
superscript timestamp "01:23" tucked under the wordmark in the
monospace, suggesting the timestamped-citations idea without
shouting it. No icon shape — just type.
- **Empty-state illustration:** A simple line drawing of a play
button and a stack of three index cards beside it. Hand-drawn
aesthetic, not a flat icon.
- **Demo lecture thumbnails:** Generated via Nano Banana 2 with
prompts that capture the specific subject (cardiology lecturer at
a whiteboard with a sketched ECG; Brazilian PhD lecturer with a
blackboard of econometric formulas; Korean professor at a
projection screen showing Python code). Each demo thumbnail should
look captured-from-video, not rendered — slightly soft focus, the
faint banding of a video stream.
- **Slide-frame crops:** Real captures from the source video,
cropped tight with a one-pixel border. Never decorated, never
rendered as a fake screen.
- **Speaker avatars:** Greyscale circular crops from the first
identified appearance of each speaker. Optional; only on
multi-speaker panels. Never invented when the speaker's face is
not on camera.
- **Stock fallbacks:** If image generation fails, fall back to
`/public/samples/sample-lecture-thumbnail.jpg`. Never to a "🎓"
emoji.
- **Generated imagery:** prefer Nano Banana 2 over stock photography.
Prompt for warmth, asymmetry, and slight imperfection — avoid the
glossy 'AI render' look.
- **Optimisation:** WebP/AVIF, `loading="lazy"`, explicit `width` /
`height` to prevent layout shift. Video player is `loading="lazy"`
until the user scrolls into view.
- **Icons:** `lucide-react` for UI. Use sparingly — never
decorative-only.
### Build-time asset manifest (explicit specs)
Every image, illustration, and visual reference mentioned above must resolve to ONE of the three buckets below — runtime-generated, seed-shipped, or user-supplied. Do NOT ship `` tags whose `src` is not listed here. Do NOT depend on bare "section 8a prompts" without binding them to explicit paths and model IDs.
**Bucket 1 — Runtime-generated (Nano Banana Pro `gemini-3-pro-image` for hero/demo photographs; Nano Banana 2 `gemini-3.1-flash-image` for in-app illustrations and reference-conditioned variants).** Cached to Firebase Storage; served via signed URL. Every reference above to "Nano Banana 2" or "Nano Banana Pro" MUST be wired to one of these specific calls with an explicit model id:
- `/public/generated/hero.webp` (2400×1500, WebP) — model `gemini-3-pro-image` — uses the literal prompt described as "Hero image (landing screen)" above. Run once at build; commit a `/public/samples/hero-fallback.webp` (1600×1000) generated from the same prompt with `gemini-3.1-flash-image` so the page renders if quota is exhausted.
- `/public/generated/demo/{demo-slug}-{NN}.webp` (1600×1200, WebP) — model `gemini-3.1-flash-image` (reference-conditioned where the prior frame is passed as input) — one path per "Demo X" image referenced above. The slug derives from the seed example in section 8a; the NN index covers each frame in the demo sequence.
- `/public/generated/illustrations/{name}.webp` (1024×1024, WebP) — model `gemini-3.1-flash-image` — one path per named illustration above ("Empty-state illustration", "Recipe-card hero illustrations", "Curriculum picker imagery", "Period-style frames", etc.). Each illustration's prompt is the literal description above; ship a deterministic seed in the request so re-runs are reproducible.
**Bucket 2 — Seed assets shipped with the deliverable.** Every "Stock fallback" path referenced above (e.g. `/public/samples/sample-X.jpg`) is generated once via Nano Banana 2 (`gemini-3.1-flash-image`) at 1024×1024 WebP using the same prompt as its Bucket-1 counterpart, then committed to the repo so the page renders identically if Gemini quota is exhausted or the user is offline. Replace any `.jpg` extension above with `.webp` to match the optimisation rule. Also commit these empty-state seeds (1024×1024 WebP, single-stroke hand-drawn line, no colour fill):
- `/public/samples/empty-state-primary.webp` — line drawing of the app's primary empty surface (the named "Empty-state illustration" above), generated from that exact prompt.
- `/public/samples/empty-state-archive.webp` — line drawing of an empty saved/archive view, single-stroke outline.
- `/public/samples/empty-state-error.webp` — line drawing of a hand placing a single object aside with care, used when an AI call fails.
**Bucket 3 — User-supplied.** Uploads from the user's camera / file picker land at the Firebase Storage path conventional for this template (named in section 4b). The build ships with Bucket-1 + Bucket-2 only; no user-supplied images at first paint.
**Hard rules**
- Every `` tag MUST have a `src` that resolves to a path listed in Bucket 1, Bucket 2, or a Bucket 3 upload path. Anything else is a build error.
- No bare `image.jpg` / `hero.jpg` / `placeholder.png` references anywhere in the code.
- Model IDs: `gemini-3-pro-image` for hero-quality photographic generation; `gemini-3.1-flash-image` for in-app illustrations, reference-conditioned variants, empty-state seeds, and stock fallbacks. Never use a legacy model id (no `imagen-*`, no `gemini-1.5-*-image`).
- File format: WebP everywhere (AVIF acceptable where the target browsers support it). No `.jpg` / `.jpeg` / `.png` in `/public/samples/`.
## 10. Interactivity & states
- Every interactive element has hover, focus, active, and disabled
states.
- Forms validate inline and show specific error messages (not
"Invalid input"). URL paste box validates each URL on blur and
shows "this looks private — paste an unlisted link or upload the
file" inline when a private YouTube URL is detected.
- Loading states use skeletons that match the eventual layout, not
spinners. The processing-queue progress steps are honest about
what's happening at each stage, not "Working…".
- Empty states explain the next action with a button whose label
fits THIS app's domain: "Paste a YouTube URL to start", "Upload a
video file", "Open a sample course" — never a generic "Add your
first item".
- Smooth scroll for in-page anchors. Clicking a bullet's timestamp
smoothly scrolls the video player into view and jumps the
playhead.
- All AI-generated content streams in token-by-token where supported.
The outline view shows sections appearing one at a time as the
model produces them, with a clear "thinking…" indicator before
content starts arriving.
- If an AI call fails, show a calm, specific error ("We couldn't
read this video — the audio drops out around 00:42. Want to upload
a different recording, or skip this lecture?") and offer retry.
- Low-confidence bullets are faintly amber-flagged; tapping reveals
the alternates the model considered and a "listen yourself"
button that jumps the video to the passage.
- The re-watch player's auto-advance takes 700 ms with
`prefers-reduced-motion` falling back to instant.
- Keyboard shortcuts: `j` / `k` move between bullets; `Enter` jumps
the video to the bullet's timestamp; `space` adds a bullet to the
card queue; `r` marks the bullet's window for re-watch;
`?` opens the keyboard help; `/` focuses the search box.
## 11. Tech & responsive requirements
- **YouTube URL pre-flight check:** before sending a YouTube URL to Gemini, run a HEAD/oEmbed pre-flight that catches 404, private, geo-blocked, embed-disabled, and age-gated videos. On any failure, fall back to: prompt the user to download an `.mp4` of the relevant segment and upload it via the Files API (returns a `files/*` resource name).
- **TTS markdown-stripping preprocessor:** before sending any user-authored markdown to `gemini-3.1-flash-tts-preview`, strip non-spoken markdown: `#`/`##`/`###` headings (keep the title text), `**bold**` (keep the inner text), `[label](url)` (keep `label`, drop URL), `` ``` `` fenced code blocks (skip entirely), `>` block-quote markers (keep the text), and `|` table pipes (read row-by-row as sentences). Insert `…` between sentences for a short pause and a blank line plus `—` between paragraphs for a long pause. The model does not understand markdown; raw markdown will be read aloud as literal characters ("asterisk asterisk").
- **File downloads on Safari / Firefox:** when offering local-disk save of any export (PDF, CSV, MP3, ZIP, JSON, image), fall back to `` with a blob URL — the File System Access API (`showSaveFilePicker()`) is Chromium-only. Detect with `'showSaveFilePicker' in window`; otherwise use the anchor-download path.
- **Stack:** React + TypeScript + Tailwind CSS. Functional components
+ hooks. Use Shadcn UI primitives where appropriate.
- **Build runtime:** AI Studio Build — full-stack with Cloud Run
server-side functions. All Gemini API calls happen server-side;
API key lives in Secrets Manager, never in client bundle.
- **Model selection:** explicitly pin `gemini-3.5-flash` for the
per-video parse, translate, flashcard, re-watch ranking, course-wide
concept linking, and "did the lecturer say X?" calls. Pin
`gemini-3.5-flash` for slide-frame descriptions. Pin
`gemini-3.1-flash-tts-preview` for audio summaries. Set
`thinkingLevel` explicitly per call per the matrix.
- **Database:** Firestore (auto-provisioned by AI Studio Build).
Show the seed course on first launch.
- **Auth:** Firebase Auth — Google sign-in by default; Apple sign-in
next to it; magic-link email as fallback for sharing.
- **Storage:** Firebase Storage for uploaded video files. Pre-signed
URLs only.
- **Video processing:** server-side `yt-dlp` (or YouTube Data API
for metadata only) for public YouTube URLs; direct Files API
upload for user-uploaded files. **Never** download a private
YouTube video without the uploader's consent.
- **Mobile-first.** Verify layouts at 375 px (iPhone SE), 768 px
(iPad), 1024 px, 1440 px+.
- Use `clamp()` for fluid typography. Prefer container queries over
media queries for component-level responsiveness.
- Use `dvh` / `svh` instead of `vh`. Respect safe-area insets on iOS.
- Zero horizontal overflow at any width. Zero layout shift on load.
- Persist user data in Firestore. Use real-time listeners on the
processing queue (so the user sees lectures complete as they
process).
- Optimistic UI on writes; reconcile on response.
- The video player uses native HTML5 `