How It Works — Written Guide

The site is a self-paced AI-engineering curriculum delivered as a single Next.js App Router application deployed to Vercel. Every lesson, from "Phase 1" fundamentals to the production‑ship phase, is a plain Markdown file under content/ — for example, content/phase-2-prompting.md. There is no database of lesson text at rest; the filesystem is the authoritive store.

The architecture’s backbone is one ordered list: LESSON_SLUGS in lib/articles.ts. Each slug’s position in that array (1‑indexed) defines its lesson number. The same array also drives the audiobook auto‑advance sequence — the “flow order” that the persistent player follows. Because the array is used for both numbering and sequencing, the two are always in sync: lesson 4 follows lesson 3 in the player because the slug at index 4 follows the slug at index 3. This single source of truth eliminates drift between a separate numbering list and a separate play‑order list. LESSON_NUMBER is a computed lookup object derived from the same array, so every getLessonBySlug call can retrieve the correct number without a second config.

The lesson read path lives in lib/data.ts and implements a three‑tier fallback designed for a Vercel‑deployed app that may or may not have a D1 database attached. On each read (getLessonBySlug), the first tier attempts to fetch from a D1 content‑cache — a writable database that lets authors push updated lesson content without redeploying the entire site. If D1 is unavailable or empty, the second tier loads from a JSON bundle (export-content) that was generated at build time from the same Markdown files. This bundle is fast and cacheable but static. The final fallback is the filesystem parser in lib/articles.ts itself, which reads the .md files directly at runtime — useful during local development or when no build output exists.

Why not just rely on one tier? Each tier buys a different operational property. D1 gives hot‑content updates without a build — ideal for corrections or additions between deployments. The JSON bundle makes cold starts fast and survives D1 outages. The filesystem parser keeps the app runnable out of the box on a fresh clone without any infrastructure setup. The fallback chain is expressed with a simple pattern: try the fast path, catch silently and move to the next. The same three‑tier pattern is used for getAllLessons, getCategoryMeta, and getGroupedLessons. This layered approach means the site can operate in environments ranging from a fully‑provisioned production Vercel deployment, through preview branches with D1 replicas, down to a local npm run dev with nothing but a content/ directory.

The Lesson interface itself is lean: slug, number, title, category, excerpt, difficulty, wordCount, readingTimeMin, url. The difficulty field is never stored — it is derived positionally by getDifficulty in lib/articles.ts, which slices each category’s lesson range into beginner, intermediate, and advanced bands. This keeps the data model immutable and avoids stale difficulty metadata. The entire curriculum — numbering, ordering, difficulty classification, and audiobook spine — flows from that single LESSON_SLUGS array, a design that trades flexibility for coherence and makes the player’s auto‑advance both predictable and trivially debuggable.

The build pipeline begins in package.json, where the prebuild script runs before build. It invokes tsx scripts/prebuild.ts, a gate that executes a battery of content‑contract tests—among them test:audio:all and test:linkedin—so that any broken assertion aborts the process before an artifact is produced. This is the central principle: gate the build, not the deploy. If a content error slips past local development, it is caught here, not after publication.

The build command itself is next build --webpack. The --webpack flag is mandatory because the production bundle must embed markdown registries as raw strings. The next.config.ts file defines a webpack rule using asset/source that matches paths under data/app-prep, specs/case-study/deep-dives, and other directories. Deployed serverless functions cannot fs.readFile repository files at runtime—the filesystem is read‑only—so these sources must be statically inlined during compilation. Without that rule, the production build would silently produce empty pages.

Deployment is not a simple vercel deploy --prod. The package.json defines deploy as bash scripts/deploy.sh, and this script is the only mechanism used to ship to Vercel. The reason is structural: the app is a git submodule inside the v9ai/ai-apps monorepo. A standard Vercel upload performs a git‑aware transfer that skips submodule contents, yielding a broken deployment. The script circumvents this by building locally (files exist on disk) and then shipping a single .tgz archive with vercel deploy --prebuilt --prod --archive=tgz. That archive also bypasses the 5000‑file upload cap of the free tier (api-upload-free).

The script must run from the monorepo root, not from the app directory. This is driven by next.config.ts, which sets outputFileTracingRoot to path.join(__dirname, "../..")—the monorepo root. The Next.js builder resolves traced dependencies relative to that root; running from apps/ai-engineer-roadmap would cause it to look for paths twice (e.g., apps/ai-engineer-roadmap/apps/…), triggering a failure after a successful build. The script therefore changes directory to the monorepo root at the start of its execution.

Several guardrails are embedded in scripts/deploy.sh. Before any build step, it reads the buildCommand key from the app’s vercel.json and verifies it is exactly "pnpm run build". A background content loop sometimes rewrites that value to a broken turbo filter; the guard fails fast with a clear message, preventing a confusing mid‑build crash.

After the deployment, the script verifies the apex domain ai-engineer-roadmap.xyz. It does not trust a bare HTTP 200—any server can return that. Instead, it sends a cache‑busted request and checks for the x-vercel-id response header, which only a Vercel edge sets. If the header is missing, the script prints the exact remediation: vercel alias set <deploy_url> ai-engineer-roadmap.xyz and exits with a failure. Only when the header is present and the status code is 200 does the script succeed.

The entire pipeline—from prebuild through the deploy script’s own checks—ensures that a broken artifact never reaches production. The build is gated by tests, the build itself embeds required assets, and the deploy script validates both the build command and the final alias. Every failure surfaces before any user sees a stale or empty page.

Most guide pages are generated not by writing from memory but by retrieving relevant fragments from the actual source code. This content‑grounding pipeline lives in roadmap‑kg/kg/memory_common.py. At its core is build_cached_index, the drop‑in replacement for an ad‑hoc VectorStoreIndex.from_documents(…). Instead of embedding everything from scratch each time, it builds a persistent, searchable index over the central store — the same Qdrant collection that the “Explain” service uses, with incremental sync by ref_doc_id so unchanged documents skip split/embed/insert entirely.

Before documents enter the index they are chopped into manageable segments. The pipeline uses a _default_splitter – a SentenceSplitter from LlamaIndex – that cuts each document into chunks of tokens. Every chunk overlaps with its neighbours, so a fact that straddles a chunk boundary (e.g. a sentence that ends one chunk and continues in the next) is still captured in full by at least one chunk. Without that overlap, the retrieval step would lose the second half of the thought, and the generated explanation would be missing the conclusion. The chunking behaviour is configurable via the splitter parameter that build_cached_index accepts; callers can supply a custom splitter, and the defaults are what every lane in the app uses by omission.

Once chunked, the documents are handed to sync_documents, which computes a content‑keyed synthetic ref_doc_id for each. An unchanged document produces the same id, so the sync layer skips split/embed/insert entirely — only new or edited documents are processed, and edited documents are replaced in place. The sync writes the resulting vectors and payloads into the namespace’s STABLE Qdrant collection. Because the index is scope‑filtered (_attach_scope), each lane retrieves over exactly the documents it passed, even though the physical collection is shared.

If Qdrant is unavailable (another process holds the lock, or VECTOR_STORE=memory is set), build_cached_index falls back to an in‑memory VectorStoreIndex. The fallback is loud when Qdrant was intended; it hard‑fails under REQUIRE_QDRANT=1. But the deliberate opt‑out for hermetic tests or benchmark lanes stays quiet. In either case, the result is a VectorStoreIndex whose retrieval is grounded in real source text, not author memory.

Chunking plus overlap is the precondition for that grounded generation. A single long‑form explanation often weaves several facts together; if a key sentence fell on a chunk boundary, the retriever would never see its conclusion, and the generated text would invent a gap. The overlap guarantees that every sentence belongs to at least two chunks, so the full context survives into the vector index. The entire pipeline — _default_splitter, build_cached_index, sync_documents, the Qdrant store — is designed so that every guide page is a retrieval‑augmented generation over the real codebase, not a hand‑written fiction.

The embedding strategy powering the guide’s memory lanes and the “Explain this” service is entirely local — no embedding API key exists anywhere in the stack. The model is BAAI/bge-small-en-v1.5 (384 dimensions) from the FastEmbed family, downloaded once as ONNX weights and then run in‑process. The function kg.llm.make_embed constructs this model; because the weights live on disk and the inference engine runs locally, every embedding call is a local computation with zero network cost. The constructed model is cached for the life of the process keyed by the EMBED_MODEL environment variable (defaulting to that BAAI identifier), so repeated calls to make_embed inside a single run return the same object — a pattern called a “process‑cached FastEmbed model” that makes per‑text embedding nearly free after the first call.

Even though the model itself is fast, a pipeline that re‑embeds an entire corpus on every grounding run would still waste CPU. The persistent cache in roadmap‑kg/kg/memory_common.py solves that. Every text that gets embedded passes through embed_nodes_cached (the shared helper for memory lanes) or the lower‑level cached_text_embeddings. Each text is hashed together with the active model name using embed_cache_key (a deterministic SHA‑256 of model + "\x1f" + text), and the resulting vector is stored in a local SQLite database at .embed‑cache/embeddings.db (configurable via EMBED_CACHE_DB or MEMORY_EMBED_CACHE_PATH). The cache is incremental: editing one chapter re‑embeds only that chapter’s text, leaving every other cached vector untouched. De‑duplication within a single call also prevents identical texts (e.g. repeated prompts) from being embedded more than once.

This design buys two critical properties. First, the pipeline runs with the network unplugged: no API calls, no keys to rotate, no rate limits. Second, re‑running the same corpus (or a largely unchanged one) costs near zero — the cache satisfies almost every lookup, and only newly introduced or edited texts trigger an actual embedding. The environment variable EMBED_CACHE=0 can disable the persistent cache for benchmarks, but in normal operation the combination of an in‑process model and a one‑time‑per‑text SQLite store keeps the whole system local and fast.

The language model is reached through make_llm in roadmap-kg/kg/llm.py, which resolves exactly one of three egress tiers in strict first‑available‑wins order. The function never tries multiple tiers in sequence; instead it returns as soon as it finds a configured path, and the choice is locked at startup so every request in the process uses the same tier. This fixed‑precedence design prevents a subtle class of bugs that “try whatever is around” heuristics suffer from—for example, a developer who sets DEEPSEEK_API_KEY for production but inadvertently leaves a LLM_BASE_URL lying around in a .env file would silently send traffic to a local server rather than the intended DeepSeek API. By checking the direct key first and returning immediately, the code guarantees that a stray LLM_BASE_URL cannot quietly pull a request onto a local model.

Tier 1 – DeepSeek’s own API (highest precedence).
When the environment variable DEEPSEEK_API_KEY is set, make_llm builds a DeepSeekLLM (an OpenAILike subclass defined in kg/deepseek_llm.py) pointed at api.deepseek.com. The model id is read from DEEPSEEK_MODEL_DIRECT (default: deepseek-v4-flash), and if the legacy model deepseek-chat is still set, it is transparently mapped to its successor deepseek-v4-flash. The key is sourced out‑of‑band (one‑shot env or an out‑of‑repo env file) and must never be persisted in the public repo’s .env, because the auto‑commit bot could stage it. This tier enables native function calling (DeepSeekLLM defaults is_function_calling_model=True), which means agents can drive tools natively without falling back to ReAct. The context window defaults to one million tokens (adjustable via DEEPSEEK_CONTEXT_WINDOW), and the completion length can be capped with max_tokens or the LLM_MAX_TOKENS env override.

Tier 2 – Local OpenAI‑compatible server.
If no DEEPSEEK_API_KEY is present but LLM_BASE_URL is set (stripped of trailing slashes), the code instantiates plain OpenAILike against that local server—typically llama.cpp’s llama-server or LM Studio. Because quantized GGUF models tool‑call unreliably, this tier keeps is_function_calling_model=False; agents fall back to ReAct for tool use. An extra body with {"stop": …} is injected when needed (e.g., for JSON‑mode extraction). The local path is intentionally not qualified as a fallback—if the key is absent and no LLM_BASE_URL exists, the code does not fall through to a default localhost; it goes to tier 3 instead.

Tier 3 – Cloudflare AI Gateway.
The last resort calls DeepSeek through the Cloudflare AI Gateway’s OpenAI‑compatible /compat endpoint. The base URL defaults to gateway.ai.cloudflare.com/v1/a036f50e02431c89170b8f977e982a3d/ai-gateway/compat (overridable via CF_AIG_BASE_URL). The model id is provider‑prefixed with deepseek/ by _gateway_model() so the compat surface routes correctly; the default model is deepseek-v4-pro (configurable through DEEPSEEK_MODEL). The gateway token is read from CF_AIG_TOKEN and the function panics with RuntimeError if it is missing. This tier inherits the same DeepSeekLLM class and native function‑calling capability as tier 1, but the endpoint is routed through Cloudflare’s infrastructure for latency monitoring and fallback.

Across all tiers, the timeout and retry count are tunable per tier via env overrides (DEEPSEEK_TIMEOUT, DEEPSEEK_MAX_RETRIES for the DeepSeek tiers; LLM_TIMEOUT, LLM_MAX_RETRIES for the local server). The _selftest in kg/llm.py verifies the per‑call timeout and retry resilience layer offline, without burning a real API call. The final call graph is simple: after make_llm returns, configure_settings assigns the chosen LLM to llama_index.core.Settings.llm, so every downstream component—property graph extraction, retrievers, query engines—uses that one LLM and never demands an OPENAI_API_KEY.

Generating a code excerpt from a source file sounds like a safe operation: the model sees the code and returns a snippet of it. In practice, a language model can produce an excerpt that looks perfectly plausible but quietly invents an API call, a function name, or an import that does not exist in the real source. The ground pipeline therefore pairs generation with a verification step that checks every identifier in the output against the actual source, accepting the excerpt only when it passes.

The verification starts with _source_identifiers inside roadmap-kg/kg/ground.py. This utility reads every source file in the chapter’s list, tokenizes the raw text into a set of whole identifiers. It splits on any non-alphanumeric character (whitespace, punctuation, operators) so that a name like index_documents becomes a single token. It also strips Python keywords (def, class, return, etc.) and built‑in names (print, len) to avoid false negatives — a hallucinated print call should not be flagged as an invention because the source already uses it. The crucial design choice is whole‑token membership: the source may contain the identifier index_documents; if the model writes documents or index as a standalone token, those are not present in the source’s identifier set and will be counted as violations. This prevents the model from sneaking in a fake name by using a substring of a real one.

Once the excerpt is generated, _code_grounding_violations enforces the bar. It extracts every identifier from the generated code (again tokenizing the same way) and checks each one against the reference set. Any identifier not found is a violation. If violations exist, the pipeline does not ship the excerpt. Instead, it retries the generation — typically with a slightly modified prompt that emphasizes verbatim copying — up to a configurable limit. The constant that governs this limit is not shown in the excerpt but the code lane uses the same retry logic as the audio lane; a chapter that fails repeatedly is simply dropped. The chapter’s slot in the output file becomes empty or omitted, and the page renders without a code block for that topic.

This trust‑but‑verify approach buys a hard guarantee: every code block that reaches the page is composed solely of identifiers that appear in the real source files. The trade‑off is a small chance of rejecting a legitimate excerpt when the model uses an uncommon but valid identifier that happens to be a substring of a larger name — for example, if the source contains database_url and the model writes url as a variable name, that url token is absent from the source set and the excerpt is flagged. In practice, the strict whole‑token rule catches far more hallucinations than it falsely rejects, and the retry loop handles the borderline cases by giving the model another chance with clearer instructions. The result is a pipeline that ships code with confidence, because every shipped token has been verified against the ground truth.

Every guide on this site is listenable, yet none of the audio is recorded by hand. The narration is synthesized from the same text the written page shows, driven by a three-stage pipeline in roadmap-kg/kg/audio/pipeline.py. That module is a convenience wrapper that runs the three stages in order: tts, finalize, and stitch. The first stage, text-to-speech, runs per chapter (or per paragraph, depending on the --per-chapter or --per-paragraph flag). It produces chapter‑level MP3s that are uploaded to R2 via --upload. The second stage, finalize, writes the page‑served data/<slug>.json — an AudioMeta object containing per‑chapter start seconds and the full‑length MP3’s metadata. The third stage, stitch, concatenates those chapter MP3s into one full‑length file on R2, again using the --upload flag. The pipeline is designed so that each stage is independently runnable (python -m kg.audio.pipeline --slug <slug> --per-chapter --upload), and re‑synthesising a single chapter is possible via --chapter N while still running finalize and stitch for the whole. This automation buys atomicity: the audio and the written guide always agree because both derive from the same source text, and a broken manifest is caught by the prebuild test suite (test:audio:all) before the bundle ever ships.

The resulting MP3s live on R2 under the public domain tts.vadim.blog rather than in the app bundle. Storing large binary assets outside the Next.js bundle keeps the worker’s memory footprint small and allows the audio to be served from a separate origin without bloating the deploy. However, serving media cross‑origin is fragile: CORS headers are needed, and all bytes must pass through the browser’s service‑worker fetch handler, which historically stalled playback by downloading the entire file. The site sidesteps both problems by streaming the MP3s through its own same‑origin proxy, defined in app/api/audio/[...path]/route.ts. This route runs on Vercel serverless (Node.js runtime, force-dynamic) and proxies requests from paths like /api/audio/knowledge/<slug>/NN.mp3 to the upstream R2 URL. The key trick: it forwards the browser’s Range header upstream and faithfully relays the 206 Partial Content (or 200, 416) response, along with Accept-Ranges, Content-Range, and Content-Length. This lets the <audio> element seek normally by mapping time to byte offset. The proxy is locked down by a regex (KEY_RE) that only permits knowledge/…mp3 paths, preventing it from becoming an open SSRF vector. It also retries transient upstream failures once with a 250ms backoff.

The player on the page obtains the same‑origin URL via toSameOriginAudioUrl, defined in app/how-it-works/_shared.ts. That helper transforms an R2 public URL (e.g. https://tts.vadim.blog/knowledge/how-it-works-audio/full.mp3) into /api/audio/knowledge/how-it-works-audio/full.mp3. Because the MP3 is now fetched from the same origin as the page, no CORS is required, and the service worker, which registers no fetch handler and self‑unregisters, cannot intercept the stream. Playback progresses byte‑by‑byte via HTTP Range requests rather than buffering the entire file. Logging in the route emits one line per request (key, range, upstream status, length, elapsed) visible in the Vercel runtime logs, making it straightforward to debug seeking or latency issues. The result is a seamless listening experience that stays same‑origin by design, with no manual recording and no fragile cross‑origin setup.

The runtime data plane exists to make new or edited content appear on the live site without a redeploy. Rather than shipping a static build that bakes every lesson and audio guide into the container, the app reads dynamic records from a small Cloudflare D1 database. The database is reached from Vercel’s Node.js runtime over Cloudflare’s REST API: every call to d1Query constructs an INSERT/SELECT statement and POSTs it to api.cloudflare.com using the credentials that d1Configured validates — this means the app only tries D1 when CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_AUDIO_D1_ID, and CLOUDFLARE_D1 are set. The store itself is a single table, content_cache, keyed by (kind, slug). Each row’s payload column holds the exact JSON that the rest of the app already consumes — a ContentIndex, LessonFull, or AudioMeta blob. Reading is deliberately boring: getDoc<T> runs a SELECT payload … WHERE kind = ? AND slug = ? LIMIT 1, calls JSON.parse on the result, and hands the typed object back to the caller. Because the payload matches the existing shapers (lessonWithContentFromFull, etc.), no transformation logic needs to duplicate the static pipeline.

Shared content reads — the index, every lesson, every audio meta — opt into the Next Data Cache by passing { revalidate: 3600, tags: [kind, kind+':'+slug] } to d1Query. This one-hour TTL is a deliberate trade‑off: uncached reads inside the top‑level render path force that entire route to be dynamic (and thus Vercel‑paid per request), so the cache prevents every visitor from hitting D1 directly. The tag pair (kind plus the more specific kind:slug) lets the offline publish pipeline later bust exactly the right rows. The /api/revalidate route (in app/api/revalidate/route.ts) is an authenticated endpoint that the kg.sync_d1 step calls after writing new rows. It loops over the posted tags and calls revalidateTag(t, "max"), which evicts the cached entries from the shared Data Cache. Within seconds the next request fetches the fresh row from D1.

Per‑user or mutating callers (though none appear in the provided consumer paths) would pass no-store to d1Query’s options object, because caching a personalized response would be incorrect. The audio media itself lives on R2 behind tts.vadim.blog. It is proxied same‑origin through the app/api/audio/ route, which forwards the browser’s Range header and relays the upstream 206/200 response. This removes the cross‑origin CORS fragility and lets <audio> seek directly, rather than downloading the whole file.

The entire design fails open. getDoc catches every error — D1 unavailable, network failure, missing table — and returns null. The resolver chain in lib/data.ts (which aggregates content-d1, content-json, and the static articles module) checks for null and falls through to the build‑time JSON export, and then to the static markdown parser. A lesson page never hard‑depends on D1; the database is an optional fast path for live updates. The same pattern applies to the content index, category metadata, and audio guides: if D1 is unconfigured or empty, the app silently degrades to what was bundled at build time. This ensures that a missing database credential, a region outage, or a table rebuild never breaks the site — it only pauses dynamic content delivery until D1 recovers.

Every scheduled job on this site runs as a Vercel cron, not as a Cloudflare Worker scheduled event. The infrastructure is unified: one Next.js App Router application ships to Vercel, and vercel.json declares a single scheduled task in its crons array: { "path": "/api/cron/backup", "schedule": "0 4 * * *" }. This means Vercel’s built-in cron system issues an ordinary HTTP GET against the app/api/cron/backup/route.ts endpoint once daily at 04:00 UTC. Because the route is a standard Next.js API handler, it inherits the application’s full runtime (database connections, environment variables, serverless function settings) without needing a separate worker runtime.

The backup route first authenticates the caller using verifyCronSecret, a function imported from @ai-apps/db-backup. Vercel’s cron injects an Authorization: Bearer header whose value must match the CRON_SECRET environment variable; if the header is missing or incorrect, the route returns a 401. This ensures the public URL is not an open trigger. Once authenticated, the handler runs runD1Backup and runQdrantBackup, writing snapshots to R2. The function’s maxDuration is set to 300 seconds in vercel.json under the functions key, giving the backup ample time to complete on Vercel’s serverless infrastructure.

A second scheduled route, app/api/cron/kg-tick/route.ts, shares the same authentication pattern and is designed to run the autonomous knowledge-graph loop. However, it is deliberately left unscheduled — there is no corresponding entry in vercel.json’s crons. Instead, it contacts the KG service on demand only when KG_TICK_ENABLED is set to "1". The default state is off, so wiring a live LLAMAINDEX_SERVICE_URL cannot silently start recurring token consumption. The route itself supports both GET (the cron verb) and POST for manual triggering, but it is never invoked automatically.

Beyond the Next.js application, a handful of Cloudflare Workers exist separately: edge-tasks, aer-ext-api, ai-gateway, and vadim-blog. These are independent services — they are not part of the Vercel app and have no cron triggers. Earlier versions of workers/edge-tasks/wrangler.toml included a [triggers] section with a */10 * * * * cron to keep a Render-hosted RAG service warm, but that target was suspended and the cron was removed. The workers now serve only request-driven tasks (e.g., proxying audio streams or calling DeepSeek directly), while all recurring maintenance — backups, knowledge-graph ticks — lives inside Vercel’s cron ecosystem.

Machine-generated audio guides cannot be blindly trusted: the synthesis pipeline emits files that may be structurally malformed, contain invalid metadata, or include sidecar artifacts like *.tts.json, *.eli5.json, and glossary catalogs that are never meant to be published as audio guides. Without a mechanical gate, these would all surface as generic failures and bury the real content issues the editorial loop must fix. That is why roadmap-kg/kg/audio_gate.py exists — a standalone, deterministic validator that runs over each generated AudioMeta JSON and returns a clean pass/fail verdict (exit 0 or 1) with no hidden state.

The gate is intentionally self-contained: it pulls in no LlamaIndex, embedding, or environment machinery, staying cheap enough to run on every guide without slowing the generation pipeline. Its entry point is gate_audio(meta: dict) -> dict, which mirrors the Rust parity oracle gate.rs. The module is invoked via python -m kg.audio_gate with either a single --slug (for per-file gating) or --all to scan a whole directory. In single‑slug mode the exit code tells the pipeline immediately whether to proceed to the next stage.

The gate splits its checks into two categories to avoid false-positive noise blocking valid content. HARD failures directly set ok = False and block publication. These include structural requirements like at least MIN_CHAPTERS (3), a minimum total word count (MIN_TOTAL_WORDS) and per‑chapter minimum (MIN_WORDS_PER_CHAPTER), titles no longer than MAX_TITLE_WORDS (5) and MAX_TITLE_CHARS, a sentence‑length coefficient‑of‑variation floor (SENTENCE_CV_MIN), and a Flesch Reading Ease floor enforced by _scan_audio_readability. Every HARD rule is mirrored in the Rust oracle and cannot be changed unilaterally. WARN-only heuristics — acronym‑first‑use, numbers 10–19, sentence‑length variety, narration‑polarity, spoken‑list, and ai‑tells — are emitted as warnings that never flip ok, precisely because they are too false‑positive‑prone to be trusted as a gate decision. This design lets the content loop triage failures_by_rule at a glance (e.g. title_too_long×12) without opening each lesson entry.

By keeping the gate cheap, deterministic, and strict on HARD rules while reserving softer checks for the human reviewer, the pipeline avoids blocking on noisy heuristics and ensures that only genuinely broken audio guides ever reach a human’s worklist. The --skip-non-guide flag extends this pragmatism by dropping known sidecar files at validation time, so the default --all scan remains byte‑identical to the Rust oracle but the optional flag keeps the worklist clean.

Two surfaces are built within this Next.js App Router app for actually remembering what you read: the /memorize route (spaced-repetition flashcards) and the /loci route (a memory-palace trainer that walks cards along a route). Both rest on the same FSRS‑6 model from lib/sm2.ts — the shared core that computes stability, difficulty, and the next-review interval — but each surface implements its own scheduler that imports that core. The FSRS‑6 core is reused wholesale; no scheduling math is duplicated.

The loci scheduler lives in lib/loci-scheduler.ts, a file of pure functions that adds exactly two thin wrappers on top of the FSRS‑6 core. The file does not import React, use localStorage, or perform I/O — it is a stateless set of exports that any component can call. The first wrapper is an onboarding ladder: new cards graduate through three fixed short steps ([10min, 1d, 3d] by default) before free-running FSRS takes over. Every grade still calls fsrsUpdate so the latent DSR state (stability and difficulty) stays honest the whole time; the ladder only overrides the resulting interval while phase !== "fsrs". The second wrapper is an in-session Leitner queue: buildSession orders due cards along the route walk, and an again grade recycles the card to the back of the queue for another pass in the same sitting. The first grade a card receives in a session is the one persisted to FSRS — recycle passes are practice-only, never re-persisted. This rule lives in gradeSessionCard. The file exports the LociPhase union type (the four rungs: "onboard1" through "fsrs"), the PHASES read-only array, and the default ladder in minutes. The same lib/sm2.ts core is shared, but the memorize surface likely has its own scheduler with different wrappers — the shared part is only the FSRS‑6 core, not the whole scheduler.

The learning-science pages under app/how-it-works/ document the evidence base that both features apply. The merged page at app/how-it-works/learning-science/page.tsx is a fully static server component (dynamic = "force-static"). It presents a catalogue of 26 evidence-backed learning techniques (sorted by paperCount descending), each linking to DOI-verified peer-reviewed papers, and 13 memory principles hand-authored in ./content.tsx. The principles are grouped by "encoding", "practice", and "structural" groups, each with mechanism prose, evidence items with citation keys, boundary conditions, and design implications for a mnemonic-generation system. Some principles also have an mlAnalog mapping to machine-learning training mechanisms (e.g., “Prioritized experience replay”). The page includes badges showing techniqueCount, paperCount, PRINCIPLE_COUNT, and CITATION_COUNT, plus links to audio guides, a BibTeX download of all citations at /learning-science.bib, and JSON‑LD structured data describing the dataset and the principles as a collection. The technique cards use bare #slug anchors; principle cards use #principle-slug anchors. The only client island is a StartMemorizingRail funnel CTA.

Why this design? The separation of the FSRS‑6 core from the per-surface schedulers means each surface can add its own onboarding logic (like the ladder) and session mechanics (like the Leitner queue) without touching the shared scheduling math. The learning-science pages are compiled entirely at build time from static JSON (data/learning-science.json, data/technique-recommendations.json) and hand-authored ReactNode prose — no runtime queries, no API calls. This trades compile-time authoring for runtime flexibility: the evidence base is frozen at deploy time, while the scheduling logic can be tuned independently for each surface. The pure functions in lib/loci-scheduler.ts ensure that scheduling decisions are deterministic and testable, and the shared fsrsUpdate keeps the DSR state honest across both the ladder and the free-running phase. Together, these files form a system where the science (the static evidence pages) and the practice (the dynamic schedulers) are cleanly separated, each built for its own trade-off: verifiability and discoverability for the former, interactivity and personalization for the latter.

The How-It-Works guide is the hub of a small but growing family of companion guides. Alongside the central overview, there are pillar walkthroughs (first-principles, autonomy, agentic-rag, evals), primers for LangGraph, LangSmith, and LlamaIndex, a memory-science deep-dive, a stack reference, and a glossary. Every one of these pages lives in its own route under /how-it-works/ and has its own Text and (where appropriate) Audio view, yet they all share a single navigation chrome: a horizontal strip of hub pills, a per-guide Text/Audio tab bar, and a sequential Back/Next pager. The navigation is built from a single source of truth, a plain data array called CASE_STUDY_GUIDES in lib/case-study-guides.ts. That file is pure data — no JSX, no browser globals — so any server component can import it freely.

Each element of the array is a CaseStudyGuide object with fields: slug (the route segment), basePath (the route prefix for that guide), title, short (compact label for pills), icon (emoji), kind (audio or text), and two optional flags: single (a one-page guide without Text/Audio tabs, like the glossary) and textAtBase (the Text view lives directly at basePath rather than basePath/text, used for guides whose reading page predates the audio companion). This array is the registry — there is no second list of links anywhere.

The shared chrome is rendered by the CaseStudyGuideNav component in components/case-study-guide-nav.tsx. It receives the active guide’s slug and the active tab, and drives three navigational elements from the same CASE_STUDY_GUIDES array. The hub pills iterate over every guide, calling guideHref(g) to compute the correct URL. That helper uses the guide’s single and textAtBase flags to decide whether the link should point to basePath/text, basePath, or basePath directly. The current guide gets aria-current="page" and a visual highlight. The per-guide tab bar appears only if the active guide is not a single page; it renders two Link components — one for textHref (which uses guideHref(guide) logic) and one for audio at basePath/audio. The tab labels are fixed, but the URLs are derived from basePath. The Back/Next pager finds the index of the active guide in the array, then computes prev and next by indexing adjacent positions; each link’s href is again computed with guideHref. Every navigation link — hub pills, tabs, pager arrows — uses the same registry.

The purpose of this design is straightforward but crucial: one registry drives all navigation. To add a new companion guide, a developer appends one object to CASE_STUDY_GUIDES. The hub instantly includes it, the pager sequence expands, and any tab links that depend on the guide’s basePath are automatically correct. There is no need to update a separate sidebar, breadcrumb, or pager file. The risk of broken links, stale entries, or inconsistent labels is eliminated. The trade-off is that all guides must follow the same route pattern — but that consistency is exactly what makes the family navigable. The data structure is small enough to remain readable, and the component stays a server component because it receives props (activeSlug, activeTab) rather than reading search params or state. The result is a navigation system that scales to any number of companions without growing in complexity.

The hand-rolled loops that ground claims and tutor flashcards are being ported to event-driven workflows from the llama-index-workflows package, and that port adds zero new dependencies because the pinned llama-index-core already installs that standalone package; llama_index.core.workflow is a one-line shim. In roadmap-kg/kg/claim_workflow.py, the old retrieve → draft → assess → revise loop is re-expressed as @step methods connected by typed events—EvidenceEvent, DraftEvent, ReviseEvent—so the graph’s edges are declared as data rather than hidden inside imperative logic. Each step receives an event, mutates the Context via ctx.store.set, and returns the next event. The workflow runner drains the stream, and every progress event is written to the store and optionally to a JSONL trace.

Durability comes from workflow_store.py, which persists a serialized Context.to_dict snapshot after every progress event. The default store is a local-first SQLite file in .workflow-runs/runs.db, but it also supports Postgres when POSTGRES_URL is set. With a run_id, a crashed run can be resumed via --resume: the store loads the last snapshot, Context.from_dict rebuilds the running state, and only in-flight events are re-queued—completed steps do not re-execute. The store uses a simple runs table with columns for ctx_json, result_json, and timestamps, matching the same schema used by the rag_service twin.

Streaming is wired directly into the service layer: rag_service/main.py mounts a WorkflowServer under /workflows-api (gated by ENABLE_WORKFLOW_SERVER), and the run endpoint in run_routes.py (mentioned in the main file) streams ProgressEvents back to the caller. A consumer can watch the agent think in real time instead of waiting for a single blocking result.

What durable, streamable runs buy over a hand loop is the ability to pause, resume, and observe a long-running reasoning process without losing progress or redoing work. The declarative @step/typed-event pattern makes the graph explicit—every transition is visible in the code as an event type and a handler method—rather than buried in a while loop. And because the workflow lives on the same FastAPI app that already indexes lessons and serves the Next.js frontend, the /workflows-api surface is just another route, deployable alongside highlight RAG without changing the existing architecture.