Overview
Core Features
- Bring-Your-Own-Content (BYOC) Ingestion Engine: Accepts YouTube URLs, device video/audio file uploads (up to 150 MB), podcast RSS/Apple audio streams, and raw pasted text. Uses serverless ASR (OpenAI Whisper & Gemini Audio) and subtitle parsers (.srt/.vtt) to convert any media into an interactive lesson.
- Dual-Caption Interactive Player & Tap-to-Learn: Displays synchronized target-language and native-language captions with real-time auto-scrolling. Tapping any word launches an interactive two-phase AI LearnSheet displaying contextual definitions, part-of-speech, CEFR difficulty, usage examples, grammar breakdowns, and audio pronunciations.
- Context-Aware AI Sense Resolution: Leverages surrounding sentence context with Gemini 2.0 Flash & OpenAI GPT-4o to resolve the exact word sense for homophones and polysemous words (e.g., Japanese ある or Spanish llevar), eliminating context-unaware dictionary lookups.
- Spaced Repetition System (SRS) & Adaptive Drills: Features an in-app review suite powered by the SM-2 algorithm. Includes 4 drill styles (Flashcard, Cloze fill-in-the-blank, Listening comprehension, and Multiple Choice Questions) with custom distractor generation and post-drill grammar recaps.
- Multi-Language Focus & Personalization: Supports instant switching across target learning languages (Spanish, Japanese, French, German, Mandarin, etc.) with per-language vocabulary progress, daily clips, curated Discover shelves, localized TTS voice narration (Google Chirp 3 HD), and daily streak tracking.
- High-Converting Onboarding & Monetization Funnel: Includes a 14-step onboarding funnel (interactive dual-caption preview, social proof, commitment pledge, dopamine celebration step), RevenueCat paywalls/Customer Center integration, push notification scheduling (expo-notifications), and soft-gate premium upsells.
Key Engineering Achievements & Architecture
- Hybrid Tokenization & Reconciled CJK Word Segmentation: Combined browser Intl.Segmenter with custom CJK token coalescing rules (coalesceCjkWordSegments) and segmenter token timing reconciliation (reconcileWordsForLearner). This cleanly handles complex scripts (e.g., Japanese Katakana splits) without requiring expensive LLM tokenization calls for every caption line.
- Sub-Second Two-Phase AI Resolution & Caching: Engineered a two-phase serverless pipeline (word-sense → word-details) backed by global PostgreSQL cache tables (lesson-cache and word-cache) with cache versioning (PIPELINE_VERSION). Reduced API costs while delivering instant preliminary glosses and supporting request cancellation on rapid tap dismissal via AbortController.
- Multi-Tenant User Isolation & Quota Management: Built a client-side storage architecture (@immersea/users/{userId}/...) featuring multi-tenant data partitioning, legacy storage purges upon sign-in, and server-side Postgres rate-limiting RPCs (try_consume_rate_limit) with Row-Level Security (RLS) enforcement.
- Unified Cross-Platform Playback Clock: Integrated heterogeneous media sources (react-native-youtube-iframe WebView, expo-av native video, native audio, and plain-text reader) under a single playback clock contract, ensuring synchronized line-highlighting and auto-scrolling across all media formats.
Product Impact
- App Store Link: Download Immersea on the iOS App Store
