GRepoWatch
Daily at 07:00 UTC+8Subscribe中文
← Back to today’s ranking

rohitg00 /

ai-engineering-from-scratch

Learn it. Build it. Ship it for others.

Visit GitHub repository ↗
Last recorded · 2026-09-30#10

Rank in that snapshot

+855Daily Stars61,289Total Stars

Project overview

Learn it. Build it. Ship it for others.

Recorded Trending signals

rohitg00/ai-engineering-from-scratch was recorded 10 times between 2026-08-25 and 2026-09-30. Its best recorded rank is #1. The latest saved snapshot reports 855 daily Stars and 61,289 total Stars. These are archive observations, not quality ratings or continuous activity measurements.

Official project documentation

The following is an excerpt from the project’s official README, fetched from GitHub. It may change independently of this archive. RepoWatch has not installed or functionally tested this project.

Implement model internals, retrieval pipelines, and agent runtimes. Test them, inspect failures, and keep the code and evaluation results.

Start learning · Choose a path · Try a lab · Build a project · Browse the curriculum

Free, open source, MIT. Learn on the website, with a coding agent, or by running local code.

542 lessons. 20 phases. Python, TypeScript, Rust, Julia, Go.

Read in your language

🇬🇧 English · 🇨🇳 简体中文 · 🇹🇼 繁體中文(台灣) · 🇯🇵 日本語 · 🇰🇷 한국어 · 🇵🇹 Português · 🇧🇷 Português (Brasil) · 🇪🇸 Español · 🇩🇪 Deutsch · 🇫🇷 Français · 🇮🇹 Italiano · 🇳🇱 Nederlands · 🇵🇱 Polski · 🇨🇿 Čeština · 🇷🇴 Română · 🇭🇺 Magyar · 🇬🇷 Ελληνικά · 🇸🇪 Svenska · 🇩🇰 Dansk · 🇳🇴 Norsk · 🇫🇮 Suomi · 🇷🇺 Русский · 🇺🇦 Українська · 🇹🇷 Türkçe · 🇮🇱 עברית · 🇸🇦 العربية · 🇮🇷 فارسی · 🇮🇳 हिन्दी · 🇧🇩 বাংলা · 🇵🇰 اردو · 🇹🇭 ไทย · 🇻🇳 Tiếng Việt · 🇮🇩 Bahasa Indonesia · 🇵🇭 Tagalog

Sponsors

Your support keeps every lesson free and open source. See all supporters · Become a sponsor

Learning paths

RouteStarting lesson
Model foundationsSetup and tooling
LLM systemsPrompt engineering
Agents and deliveryThe agent loop

Compare career paths · Prerequisites and study time

Gradient descent

Twenty starting points follow gradient descent on a quadratic loss. The graph shows their positions and mean loss after each update.

Adjust the learning rate in the lesson · Compare GD, momentum, and Adam in code

Projects

Three projects with staged starters, reference implementations, and local graders. Run commands from the repository root after setup. Starters fail until you implement the stages.

01 · Retrieval Evaluation Lab · Python · Ranking metrics and regression checks

A candidate improves mean NDCG while one query ranks its most relevant evidence lower. Build a query-by-query comparison that reports the regression and can fail a release check.

Use Python 3.10+. Review RAG and model evaluation. Implement ranking validation, precision and recall, rank-sensitive metrics, then system comparison.

python3 scripts/project_test.py retrieval-evaluation-lab \
  --init learning-artifacts/retrieval-evaluation-lab
python3 scripts/project_test.py retrieval-evaluation-lab \
  --stage 1 --path learning-artifacts/retrieval-evaluation-lab --strict
python3 scripts/project_test.py retrieval-evaluation-lab \
  --all --path learning-artifacts/retrieval-evaluation-lab --strict

Keep: a reproducible comparison with per-query deltas and the judgments used to score them. Metrics describe those judgments; they do not establish answer correctness.

Start the project · Inspect the reference · Run on your own inputs

02 · Agent Trace Debugger · TypeScript · Trace parsing and timing

A supplied trace still takes 100 ms, but total token usage rises by 200 and one span starts failing. Separate overlapping child work from parent execution time, then produce a report that exposes the change.

Use Node.js 22.18+ and Python 3 for the grader. Implement JSONL parsing, parent validation, interval arithmetic, then an inspectable timeline.

python3 scripts/project_test.py agent-trace-debugger \
  --init learning-artifacts/agent-trace-debugger
python3 scripts/project_test.py agent-trace-debugger \
  --stage 1 --path learning-artifacts/agent-trace-debugger --strict
python3 scripts/project_test.py agent-trace-debugger \
  --all --path learning-artifacts/agent-trace-debugger --strict

Keep: the input trace, an HTML timeline, and a JSON regression report. Preserve exclusive per-span token counts so parent and child usage are not counted twice.

Start the project · Inspect the reference · Explore timing interactively

03 · Tool Call Firewall · Rust · Role checks and approval receipts

A write changes after review, or an approval is replayed. Validate the call envelope, check the caller's role and path, then consume an approval bound to the exact request and content.

Use Rust and Python 3.10+. Review tool schema design and security boundaries. The caller application supplies identity; the model proposes an operation.

python3 scripts/project_test.py tool-call-firewall \
  --init learning-artifacts/tool-call-firewall
python3 scripts/project_test.py tool-call-firewall \
  --stage 1 --path learning-artifacts/tool-call-firewall --strict
python3 scripts/project_test.py tool-call-firewall \
  --all --path learning-artifacts/tool-call-firewall --strict

Keep: an audit receipt showing the requested operation and the policy decision. Approvals are single-use within one invocation; this project does not provide persistent authorization or an operating-system sandbox.

Start the project · Inspect the reference · Explore approval boundaries

Browse all projects · Career practice guide

Choose how to learn

On the website

Open any completed lesson on aiengineeringfromscratch.com or expand a phase under Contents. No setup, no cloning.

With an AI tutor

If Node.js, npx, and a skill-capable coding agent are already installed, your coding agent can become your tutor. A repository clone is not needed to install or read the tutor. Runnable focused-path labs need python3. Agent Skills host labs also need a selected host and a writable user or project skill scope.

npx skills add rohitg00/ai-engineering-from-scratch

Choose the host and scope when the installer asks. Use start-learning in Codex, /start-learning in Claude Code, or ask your host to use the skill by name.

Tutor setup and host commands

Check the local requirements first:

node --version
npx --version
python3 --version

skills writes to the host and scope selected during installation, such as .claude/skills/, .cursor/skills/, .codex/skills/, or another supported skills folder. Verify that the selected host discovers that exact destination.

Invocation syntax belongs to the host, not to the portable SKILL.md format:

HostStart the courseStart Model Context Protocol (MCP)Start Agent SkillsRun a phase quiz
Codexstart-learning, or choose it from /skillslearn-mcp, or choose it from /skillslearn-agent-skills, or choose it from /skillscheck-understanding 13, or choose it from /skills
Claude Code/start-learning/learn-mcp/learn-agent-skills/check-understanding 13
Other compatible hostsUse start-learning to begin the course.Use learn-mcp to start the Model Context Protocol (MCP) path.Use learn-agent-skills to start the Agent Skills Engineering path.Use check-understanding to quiz me on Phase 13.

A ten-question placement quiz maps what you already know to a starting phase and saves a personalized study plan to LEARNING.md. From there, the learn skill teaches one lesson per session: concept, math, code, quiz. It streams lessons straight from this repo, and the course-guide skill jumps you to the exact lesson that covers anything you are stuck on. In Codex, invoke these skills with learn and course-guide; in Claude Code, use /learn and /course-guide; in other compatible hosts, ask to use the skill by name.

Only want Model Context Protocol (MCP)? Use the MCP invocation for your host. It creates MCP-LEARNING.md and follows one 17-lesson route through stateless requests, transports, bidirectional work, security, reliability, registry governance, and conformance evidence. The exact order and checkpoints live in the Model Context Protocol (MCP) manifest.

Only want Agent Skills? Use the Agent Skills invocation for your host. It creates AGENT-SKILLS-LEARNING.md and follows one coherent five-lesson route: contract, discovery, invocation, sandbox boundaries, then release evals and real-host portability. Start on the web with the Agent Skills path.

The installer lists the hosts it can configure and asks where to install. If you do not have Node.js, npx, python3, a supported host, or a writable scope yet, use the website or read docs/en.md manually. That path teaches the concepts, but real-host discovery, invocation, script, and uninstall evidence remains pending until the preflight is available. Read the lessons at aiengineeringfromscratch.com.

The learning skills

SkillWhat it does
start-learningOne-time onboarding: why you're learning, placement quiz, personalized plan saved to LEARNING.md.
learnThe tutor loop. Warm-up recall, then the next lesson taught interactively, then its quiz; records progress and a review queue.
course-guideTopic router. "Where do I learn attention?" or "my loss is NaN" → the exact lessons, with links.
learn-mcpFocused Model Context Protocol (MCP) tutor. Creates MCP-LEARNING.md, follows the 17-lesson manifest, and records wire, security, reliability, and conformance evidence.
learn-agent-skillsFocused Agent Skills tutor. Creates AGENT-SKILLS-LEARNING.md, teaches lessons 22, 24, 25, 26, and 27, and records real-host evidence.
learn-reverse-engineeringFocused Reverse Engineering tutor. Creates REVERSE-ENGINEERING-LEARNING.md, teaches Phase 13 lessons 32 to 45 on programs you build yourself, and records lab evidence.
claude-certificationCertification tutor. Chooses CCAO-F, CCDV-F, CCAR-F, or CCAR-P; teaches each lesson; runs labs; reviews artifacts; administers diagnostics and mocks; saves progress.
mcpa-certificationMCPA tutor. Follows the 34-lesson mcpa-f route on the 2026-07-28 protocol; teaches each lesson; runs labs and the wire checker; administers the diagnostic and three mocks; saves progress.
find-your-levelTen-question placement quiz. Maps your knowledge to a starting phase and produces a personalized path with hour estimates.
check-understanding <phase>Per-phase quiz, eight questions, with feedback and specific lessons to review. Use the Codex, Claude Code, or natural-language form in the invocation table above.

Run local code

git clone https://github.com/rohitg00/ai-engineering-from-scratch.git
cd ai-engineering-from-scratch
python3 phases/00-setup-and-tooling/01-dev-environment/code/verify.py --route beginner
python3 phases/01-math-foundations/01-linear-algebra-intuition/code/vectors.py

The preflight separates requirements needed now from tools needed later. Every required failure includes the detected reason and a corrective command. The vectors.py command is a dependency-free lesson and ends by showing that a matrix times a vector is the operation inside a neural network layer. Save that terminal output as your first evidence.

Use every lesson the same way

Use every lesson the same way

  1. Read docs/en.md and explain the core idea in your own words.
  2. Type and build the important code instead of treating the code block as decoration.
  3. Run the lesson command from the repository root, the directory containing README.md and phases/.
  4. Keep evidence: the command, working directory, exit code, meaningful output, and the artifact you changed or produced.
  5. Continue only when you can explain the output and make one small change without guessing.

Commands in lesson pages are paths from the repository root unless the lesson explicitly says to change directories. If a lesson offers several languages, run the implementation for the language you are learning.

Choose a learning path

You do not need to scan 542 lessons before beginning. Pick one goal. Each link opens the same curriculum on GitHub or the website, and both versions use the same lesson code.

Your goalLearn on GitHubLearn on the website
I am new and want the complete foundationPhase 0: Setup and ToolingDev Environment
I know Python and want math plus ML foundationsPhase 1: Math FoundationsLinear Algebra Intuition
I want to build production LLM applicationsPhase 11: LLM EngineeringPrompt Engineering
I want to build agentsPhase 14: Agent EngineeringThe Agent Loop
I want to use coding agents on real repositoriesAgent-Assisted Engineering pathAgent-Assisted Engineering
I want to shape the right build before implementationProduct Judgment and Delivery pathProduct Judgment and Delivery

Not sure where you fit? Use the start-learning placement tutor or the website prerequisites guide.

Compare four core domains and six career routes in the AI Engineering Learning Paths.

Focused MCP, Agent Skills, and Reverse Engineering paths
Your goalLearn on GitHubLearn on the website
I want to build with Model Context Protocol (MCP)Model Context Protocol (MCP) routeModel Context Protocol (MCP) path
I want to write and ship Agent SkillsFocused Agent Skills routeAgent Skills path
I want to read compiled programsReverse Engineering routeReverse Engineering path
Prerequisites and study time

Prerequisites

  • You can write code (any language; Python helps).
  • You want to understand how AI actually works, not just call APIs.

Where to start

BackgroundStart atEstimated time
New to programming and AIPhase 0 — Setup~306 hours
Know Python, new to MLPhase 1 — Math Foundations~270 hours
Know ML, new to deep learningPhase 3 — Deep Learning Core~200 hours
Know deep learning, want LLMs and agentsPhase 10 — LLMs from Scratch~100 hours
Senior engineer, only want agent engineeringPhase 14 — Agent Engineering~60 hours
Only want to build production MCP systemsModel Context Protocol (MCP) path~23 hours 15 min
Only want to build production Agent SkillsAgent Skills Engineering path~9.5 hours
Only want to learn reverse engineeringReverse Engineering path~32.5 hours

The shape of the curriculum

Twenty phases stack on top of each other. Math is the floor. Agents and production are the roof. Skip ahead if you already know the lower layers, but don't skip and then wonder why something at the top is breaking.

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#fafaf5','primaryTextColor':'#1a1a1a','primaryBorderColor':'#3553ff','lineColor':'#3553ff','fontFamily':'JetBrains Mono','fontSize':'12px'}}}%%
flowchart TB
  P0["Phase 0 — Setup &amp; Tooling"] --> P1["Phase 1 — Math Foundations"]
  P1 --> P2["Phase 2 — ML Fundamentals"]
  P2 --> P3["Phase 3 — Deep Learning Core"]
  P3 --> P4["Phase 4 — Vision"]
  P3 --> P5["Phase 5 — NLP"]
  P3 --> P6["Phase 6 — Speech &amp; Audio"]
  P3 --> P9["Phase 9 — RL"]
  P5 --> P7["Phase 7 — Transformers"]
  P7 --> P8["Phase 8 — GenAI"]
  P7 --> P10["Phase 10 — LLMs from Scratch"]
  P10 --> P11["Phase 11 — LLM Engineering"]
  P10 --> P12["Phase 12 — Multimodal"]
  P11 --> P13["Phase 13 — Tools &amp; Protocols"]
  P13 --> P14["Phase 14 — Agent Engineering"]
  P14 --> P15["Phase 15 — Autonomous Systems"]
  P15 --> P16["Phase 16 — Multi-Agent &amp; Swarms"]
  P14 --> P17["Phase 17 — Infrastructure &amp; Production"]
  P15 --> P18["Phase 18 — Ethics &amp; Alignment"]
  P16 --> P19["Phase 19 — Capstone Projects"]
  P17 --> P19
  P18 --> P19
  classDef phase fill:#fafaf5,stroke:#3553ff,color:#1a1a1a
  class P0,P1,P2,P3,P4,P5,P6,P7,P8,P9,P10,P11,P12,P13,P14,P15,P16,P17,P18,P19 phase

Contents

Twenty phases. Click any phase to expand its lesson list.

Phase 0: Setup & Tooling 13 lessons

Get your environment ready for everything that follows.

#LessonTypeLang
01Dev EnvironmentBuildPython, TypeScript, Rust
02Git & CollaborationLearn—
03GPU Setup & CloudBuildPython
04APIs & KeysBuildPython, TypeScript
05Jupyter NotebooksBuildPython
06Python EnvironmentsBuildShell
07Docker for AIBuildDocker
08Editor SetupBuild—
09Data ManagementBuildPython
10Terminal & ShellLearn—
11Linux for AILearn—
12Debugging & ProfilingBuildPython
13Python for AI EngineeringBuildPython, TypeScript
Phase 1 — Math Foundations  22 lessons  The intuition behind every AI algorithm, through code.
#LessonTypeLang
01Linear Algebra IntuitionLearnPython, Julia
02Vectors, Matrices & OperationsBuildPython, Julia
03Matrix Transformations & EigenvaluesBuildPython, Julia
04Calculus for ML: Derivatives & GradientsLearnPython, Julia
05Chain Rule & Automatic DifferentiationBuildPython, Julia
06Probability & DistributionsLearnPython, Julia
07Bayes' Theorem & Statistical ThinkingBuildPython, TypeScript
08Optimization: Gradient Descent FamilyBuildPython, Julia
09Information Theory: Entropy, KL DivergenceLearnPython, TypeScript
10Dimensionality Reduction: PCA, t-SNE, UMAPBuildPython, TypeScript, Julia
11Singular Value DecompositionBuildPython, Julia
12Tensor OperationsBuildPython, TypeScript
13Numerical StabilityBuildPython, TypeScript, Julia
14Norms & DistancesBuildPython, TypeScript
15Statistics for MLBuildPython, TypeScript
16Sampling MethodsBuildPython, TypeScript, Julia
17Linear SystemsBuildPython, TypeScript, Julia
18Convex OptimizationBuildPython, TypeScript, Julia
19Complex Numbers for AILearnPython, TypeScript
20The Fourier TransformBuildPython, TypeScript, Julia
21Graph Theory for MLBuildPython, TypeScript
22Stochastic ProcessesLearnPython, Julia
Phase 2 — ML Fundamentals  21 lessons  Classical ML — still the backbone of most production AI.
Phase 3 — Deep Learning Core  13 lessons  Neural networks from first principles. No frameworks until you build one.
#LessonTypeLang
01The Perceptron: Where It All StartedBuildPython, TypeScript, Rust, Julia
02Multi-Layer Networks & Forward PassBuildPython, TypeScript, Julia
03Backpropagation from ScratchBuildPython, TypeScript, Rust, Julia
04Activation Functions: ReLU, Sigmoid, GELU & WhyBuildPython, TypeScript, Julia
05Loss Functions: MSE, Cross-Entropy, ContrastiveBuildPython, TypeScript, Julia
06Optimizers: SGD, Momentum, Adam, AdamWBuildPython, TypeScript, Julia
07Regularization: Dropout, Weight Decay, BatchNormBuildPython, TypeScript, Julia
08Weight Initialization & Training StabilityBuildPython, TypeScript, Julia
09Learning Rate Schedules & WarmupBuildPython, TypeScript, Julia
10Build Your Own Mini FrameworkBuildPython, TypeScript, Julia
11Introduction to PyTorchBuildPython
12Introduction to JAXBuildPython
13Debugging Neural NetworksBuildPython
Phase 4 — Computer Vision  28 lessons  From pixels to understanding — image, video, 3D, VLMs, and world models.
Phase 5 — NLP: Foundations to Advanced  29 lessons  Language is the interface to intelligence.
#LessonTypeLang
01Text Processing: Tokenization, Stemming, LemmatizationBuildPython, TypeScript
02Bag of Words, TF-IDF & Text RepresentationBuildPython, TypeScript
03Word Embeddings: Word2Vec from ScratchBuildPython, Julia
04GloVe, FastText & Subword EmbeddingsBuildPython, TypeScript
05Sentiment AnalysisBuildPython, TypeScript
06Named Entity Recognition (NER)BuildPython, TypeScript
07POS Tagging & Syntactic ParsingBuildPython, TypeScript
08Text Classification — CNNs & RNNs for TextBuildPython, TypeScript
09Sequence-to-Sequence ModelsBuildPython, TypeScript
10Attention Mechanism — The BreakthroughBuildPython, TypeScript
11Machine TranslationBuildPython, TypeScript
12Text SummarizationBuildPython, TypeScript
13Question Answering SystemsBuildPython, TypeScript
14Information Retrieval & SearchBuildPython, TypeScript, Rust
15Topic Modeling: LDA, BERTopicBuildPython, TypeScript, Julia
16Text GenerationBuildPython, TypeScript
17Chatbots: Rule-Based to NeuralBuildPython, TypeScript
18Multilingual NLPBuildPython, TypeScript
19Subword Tokenization: BPE, WordPiece, Unigram, SentencePieceLearnPython, TypeScript, Rust
20Structured Outputs & Constrained DecodingBuildPython, TypeScript, Rust
21NLI & Textual EntailmentLearnPython, TypeScript
22Embedding Models Deep DiveLearnPython, TypeScript
23Chunking Strategies for RAGBuildPython, TypeScript
24Coreference ResolutionLearnPython, TypeScript
25Entity Linking & DisambiguationBuildPython, TypeScript
26Relation Extraction & Knowledge Graph ConstructionBuildPython, TypeScript
27LLM Evaluation: RAGAS, DeepEval, G-EvalBuildPython, TypeScript
28Long-Context Evaluation: NIAH, RULER, LongBench, MRCRLearnPython, TypeScript
29Dialogue State TrackingBuildPython, TypeScript
Phase 6 — Speech & Audio  17 lessons  Hear, understand, speak.
#LessonTypeLang
01Audio Fundamentals: Waveforms, Sampling, FFTLearnPython, TypeScript, Julia
02Spectrograms, Mel Scale & Audio FeaturesBuildPython, TypeScript, Julia
03Audio ClassificationBuildPython, TypeScript
04Speech Recognition (ASR)BuildPython, TypeScript
05Whisper: Architecture & Fine-TuningBuildPython, TypeScript
06Speaker Recognition & VerificationBuildPython, TypeScript
07Text-to-Speech (TTS)BuildPython, TypeScript
08Voice Cloning & Voice ConversionBuildPython, TypeScript
09Music GenerationBuildPython, TypeScript
10Audio-Language ModelsBuildPython, TypeScript
11Real-Time Audio ProcessingBuildPython, Rust
12Build a Voice Assistant PipelineBuildPython, TypeScript
13Neural Audio Codecs — EnCodec, SNAC, Mimi, DACLearnPython, TypeScript
14Voice Activity Detection & Turn-TakingBuildPython, TypeScript, Rust
15Streaming Speech-to-Speech — Moshi, HibikiLearnPython, TypeScript
16Voice Anti-Spoofing & Audio WatermarkingBuildPython, TypeScript
17Audio Evaluation — WER, MOS, MMAU, LeaderboardsLearnPython, TypeScript
Phase 7 — Transformers Deep Dive  16 lessons  The architecture that changed everything.
Phase 8 — Generative AI  15 lessons  Create images, video, audio, 3D, and more.
#LessonTypeLang
01Generative Models: Taxonomy & HistoryLearnPython, TypeScript
02Autoencoders & VAEBuildPython, TypeScript
03GANs: Generator vs DiscriminatorBuildPython, TypeScript
04Conditional GANs & Pix2PixBuildPython, TypeScript
05StyleGANBuildPython, TypeScript
06Diffusion Models — DDPM from ScratchBuildPython, TypeScript
07Latent Diffusion & Stable DiffusionBuildPython, TypeScript
08ControlNet, LoRA & ConditioningBuildPython, TypeScript
09Inpainting, Outpainting & EditingBuildPython, TypeScript
10Video GenerationBuildPython, TypeScript
11Audio GenerationBuildPython, TypeScript
123D GenerationBuildPython, TypeScript
13Flow Matching & Rectified FlowsBuildPython, TypeScript, Julia
14Evaluation: FID, CLIP ScoreBuildPython, TypeScript
19Visual Autoregressive Modeling (VAR): Next-Scale PredictionBuildPython, Julia
Phase 9 — Reinforcement Learning  12 lessons  The foundation of RLHF and game-playing AI.
#LessonTypeLang
01MDPs, States, Actions & RewardsLearnPython, TypeScript
02Dynamic ProgrammingBuildPython, TypeScript
03Monte Carlo MethodsBuildPython, TypeScript
04Q-Learning, SARSABuildPython, TypeScript
05Deep Q-Networks (DQN)BuildPython, TypeScript
06Policy Gradients — REINFORCEBuildPython, TypeScript
07Actor-Critic — A2C, A3CBuildPython, TypeScript
08PPOBuildPython, TypeScript
09Reward Modeling & RLHFBuildPython, TypeScript
10Multi-Agent RLBuildPython, TypeScript
11Sim-to-Real TransferBuildPython, TypeScript
12RL for GamesBuildPython, TypeScript
Phase 10 — LLMs from Scratch  24 lessons  Build, train, and understand large language models.
Phase 11 — LLM Engineering  17 lessons  Put LLMs to work in production.
#LessonTypeLang
01Prompt Engineering: Techniques & PatternsBuildPython, TypeScript
02Few-Shot, CoT, Tree-of-ThoughtBuildPython
03Structured OutputsBuildPython, TypeScript
04Embeddings & Vector RepresentationsBuildPython, TypeScript
05Context EngineeringBuildPython, TypeScript
06RAG: Retrieval-Augmented GenerationBuildPython, TypeScript
07Advanced RAG: Chunking, RerankingBuildPython, TypeScript
08Fine-Tuning with LoRA & QLoRABuildPython
09Function Calling & Tool UseBuildPython, TypeScript
10Evaluation & TestingBuildPython, TypeScript
11Caching, Rate Limiting & CostBuildPython, TypeScript
12Guardrails & SafetyBuildPython, TypeScript
13Building a Production LLM AppBuildPython, TypeScript
14Model Context Protocol (MCP)BuildPython, TypeScript, Go
15Prompt Caching & Context CachingBuildPython, TypeScript
16Agent State Machines — Graphs, Nodes, CheckpointsBuildPython
17Agent Framework TradeoffsLearnPython, TypeScript
Phase 12 — Multimodal AI  25 lessons  See, hear, read, and reason across modalities — from ViT patches to computer-use agents.
#LessonTypeLang
01Vision Transformers and the Patch-Token PrimitiveLearnPython, TypeScript
02CLIP and Contrastive Vision-Language PretrainingBuildPython, TypeScript
03BLIP-2 Q-Former as Modality BridgeBuildPython, TypeScript
04Flamingo and Gated Cross-AttentionLearnPython, TypeScript
05LLaVA and Visual Instruction TuningBuildPython, TypeScript
06Any-Resolution Vision — Patch-n'-Pack and NaFlexBuildPython, TypeScript
07Open-Weight VLM Recipes: What Actually MattersLearnPython, TypeScript
08LLaVA-OneVision: Single, Multi, VideoBuildPython, TypeScript
09Qwen-VL Family and Dynamic-FPS VideoLearnPython, TypeScript
10InternVL3 Native Multimodal PretrainingLearnPython, TypeScript
11Chameleon Early-Fusion Token-OnlyBuildPython, TypeScript
12Emu3 Next-Token Prediction for GenerationLearnPython, TypeScript
13Transfusion Autoregressive + DiffusionBuildPython, TypeScript
14Show-o Discrete-Diffusion UnifiedLearnPython, TypeScript
15Janus-Pro Decoupled EncodersBuildPython, TypeScript
16MIO Any-to-Any StreamingLearnPython, TypeScript
17Video-Language Temporal GroundingBuildPython, TypeScript
18Long-Video at Million-Token ContextBuildPython, TypeScript
19Audio-Language Models: Whisper to AF3BuildPython, TypeScript
20Omni Models: Thinker-Talker StreamingBuildPython, TypeScript
21Embodied VLAs: RT-2, OpenVLA, π0, GR00TLearnPython, TypeScript
22Document and Diagram UnderstandingBuildPython, TypeScript
23ColPali Vision-Native Document RAGBuildPython, TypeScript
24Multimodal RAG and Cross-Modal RetrievalBuildPython, TypeScript
25Multimodal Agents and Computer-Use (Capstone)BuildPython, TypeScript
Phase 13 — Tools & Protocols  45 lessons  The interfaces between AI and the real world.
#LessonTypeLang
01The Tool InterfaceLearnPyt