Hands-on, zero-black-box implementations of modern AI and Machine Learning fundamentals — autograd engines, BPE tokenizers, neural networks — built entirely in pure Python.
Each module builds on the previous one — from the math inside a single neuron to a quantized model served in production. All code is zero-black-box.
Part 1 — Fundamentals: the 8 topics below as
8 chapters × 5 levels, analogy to expert view, with live labs.
Part 2 — Applied: 15 chapters × 5 levels on shipping
LLM systems — structured output, tool calling, context engineering, vector DBs, RAG,
reranking, agents, MCP, deterministic pipelines, evals, guardrails, caching,
observability, fine-tuning.
Part 3 — The Python Stack (interview edition): 16 chapters × 5 levels
on NumPy, pandas, Matplotlib/Seaborn, SciPy, statsmodels, scikit-learn, XGBoost/LightGBM/CatBoost,
SHAP, Optuna, PyTorch, Keras, Hugging Face, spaCy, OpenCV, time series and MLOps.
Part 4 — ML Interviews & Production Systems: 17 chapters × 5 levels
on the interview map, probability & stats, classical ML theory, DL architectures & training,
recommenders, search/ranking, A/B testing, causal inference, ML system design, SQL, distributed
training, production monitoring & drift, and responsible AI.
Part 5 — Databases & Data Systems: 18 chapters × 5 levels
on SQL, NoSQL, vector & graph databases with production depth — data modeling, indexes &
query planners, transactions & isolation, replication, sharding, CAP/PACELC, PostgreSQL,
Redis/DynamoDB, MongoDB, Cassandra, LSM vs B-tree, the lakehouse, Elasticsearch, vector DBs,
graph DBs, Kafka/CDC, datastore selection, and DB operations — each ending in an interview drill.
Part 6 — AI/ML System Design: 16 chapters × 5 levels
on designing AI/ML systems at scale — capacity math & SLOs, online/batch/streaming serving, the
inference gateway, KServe/Triton/vLLM/TGI/BentoML, feature stores & training-serving skew, retrieval
at scale, caching for AI, GPU autoscaling & backpressure, multi-region & fallback models,
cost/perf tradeoffs, and two end-to-end walkthroughs.
Part 7 — Production Deployment & Delivery: 16 chapters × 5 levels
on shipping AI/ML — containers & reproducibility, model/artifact registries, CI for ML, GitOps CD
(Argo CD / Flux), progressive delivery (canary, blue-green, shadow traffic), Kubernetes for model
workloads, managed inference, eval-in-CI release gates, rollback & kill switches, load testing,
migrations, IaC, and supply-chain security.
Part 8 — AI Observability & Production Metrics: 16 chapters × 5 levels
on OpenTelemetry, Prometheus & Grafana, golden signals / RED / USE, SLOs & error budgets, LLM
telemetry (tokens, cost/request, TTFT, cache-hit), tracing an agent request, eval-in-prod, drift &
data quality, guardrail & safety metrics, dashboards, burn-rate alerting, AI-enriched ops, and
cost observability / FinOps.
Part 9 — Enterprise-Scale AI/ML in Production: 16 chapters × 5 levels
on platform engineering & the paved road, multi-tenancy & quotas, model & data governance,
security for AI systems, compliance (EU AI Act, NIST AI RMF, ISO/IEC 42001, SOC 2), responsible AI,
FinOps at scale, reliability & cell architecture, LLMOps, org design, vendor portability, a
reference architecture, and an enterprise-readiness checklist.
Parts 1–5 build the primitives from scratch. Parts 6–9 teach the industry-standard stack — Kubernetes, OpenTelemetry, Prometheus, Argo, Terraform, KServe — configured and operated, not reimplemented.
Build an automatic differentiation engine from 200 lines. Understand how gradients flow backward through a computation graph, then build a full MLP that reaches 100% accuracy on a toy dataset.
Build a Byte-Pair Encoding tokenizer from scratch. Understand how text becomes numbers, how tokens carry meaning, and why fewer tokens = lower bills.
Everything made touchable. A browser playground with live neuron sliders, BPE visualizer, cost calculator, and 46 automated checks over 15 real-world scenarios.
The mechanism inside GPT. Score every token against every other token, softmax the scores into percentages, and mix meanings — then train a mini transformer end-to-end with your own engine.
The operation every GPU runs billions of times per second. Build it four ways: naive, cache-tiled, Strassen divide-and-conquer, then matvec (which is exactly one attention head). Watch each algorithm agree and see why the loop order changes everything.
Meaning as geometry. Show king − man + woman ≈ queen, learn word vectors from scratch with skip-gram, then build a locality-sensitive hash that keeps ~90% recall while scoring 9× fewer vectors — the trick inside FAISS.
Everything around the model. Five prompt templates measured head-to-head, a mini-RAG that retrieves, grounds and cites (and refuses when it should), then a ReAct agent chaining lookup + calc tools with a full trace.
Changing the weights, built on the Phase 1 engine. A rank-1 LoRA patch fits a correction while training a fraction of the weights; a Bradley-Terry reward model recovers human taste from comparisons; DPO moves a policy onto preferences with no RL loop.
A zero-install browser lab — one HTML file, no server, no dependencies. Everything runs live, even offline.
Drag the sliders and watch a single neuron learn — the loss falls, the gradient flows, and the weights update in real time.
Type any text and watch the byte-pair encoding tokenizer split it, merge it, and count the tokens — and what those tokens cost.
See why fewer tokens mean lower bills — compare API costs across model sizes with realistic per-token pricing.
The famous "the animal didn't cross the street because it…" — click an ending and watch REAL attention weights re-route.
ChatGPT's creativity knob exposed: drag the temperature and watch the next-word distribution sharpen or melt.
Every token scores every token: drag the context length and watch the pairwise-score count explode quadratically.
No setup needed — everything runs locally with just Python 3.13. Follow these steps:
git clone https://github.com/satyabhan007/AI-ML.git
cd AI-MLpython lab/scenarios_tester.pylab/playground.html — no server needed.
Every concept is explained three ways — plain English, deep math, and visual diagrams.
| Guide | Topic | What You Get |
|---|---|---|
| REPORT.md | Autograd & Backprop | 1800+ line reference manual — engine internals, math proofs, examples |
| BEGINNER_GUIDE.md | Neural Nets 101 | Plain-English analogies — shower thermostat, soup tasting, golf in fog |
| VISUAL_GUIDE.md | Graph Walkthroughs | ASCII & visual diagrams of DAG backward passes and chain rule flow |
| MINDMAPS.md | Conceptual Maps | Memory aids and system maps for rapid revision |
| DEEP_EXPLANATION.md | Line-by-Line Math | Every formula broken down with worked numbers and exercises |
| Tokenizer Guide | BPE Tokenization | LEGO analogy, merge trees, compression ratios, UTF-8 handling |
| AI_ENGINEERING_NOTES.md | Learning Roadmap | 20+ project roadmap from autograd to LLMs and AI agents |
| attention.py | Attention Core | Softmax, √d scaling, causal masks, positional encoding — pure Python |
| Attention Guide | Transformers 101 | Everyday analogies for queries, keys, values — and the O(n²) bill |
| matmul.py | Matrix Multiplication | Naive → tiled → Strassen → mat-vector — the kernel behind every layer |
| MatMul Guide | GPU Kernel Intuition | Why matmul is 80% of a forward pass, and how tiling cuts memory traffic |
| Embeddings Guide | Meaning as Geometry | Cosine similarity, skip-gram training, and LSH for billion-scale search |
| RAG & Agents Guide | Using the Model | Prompt templates, retrieval + grounding + refusal, and the ReAct agent loop |
| Fine-tuning Guide | Teaching the Model | LoRA vs full fine-tune, Bradley-Terry reward models, and DPO without RL |
| Serving Guide | Shipping It | INT8 quantization, KV-cache O(n) decoding, continuous batching, roofline |
Nine courses built into this repo — Parts 1–5 from scratch, Parts 6–9 on the industry-standard production stack — then go deeper with Anthropic's official training.
Autograd → tokenization → GPUs → attention → embeddings → RAG/agents → RLHF → serving. Built from scratch, with live labs.
Open Part 1Structured output, tool calling, context engineering, vector DBs, RAG, reranking, agents, MCP, deterministic pipelines, evals, guardrails, caching, observability, fine-tuning.
Open Part 2NumPy, pandas, Matplotlib/Seaborn, SciPy, statsmodels, scikit-learn, XGBoost/LightGBM/CatBoost, SHAP, Optuna, PyTorch, Keras, Hugging Face, spaCy, OpenCV, time series, MLOps — every chapter ends in an interview drill.
Open Part 3The ML interview map, probability & stats, bias/variance, trees/SVM/kNN, clustering, feature engineering, DL architectures & training, recommenders, search & ranking, A/B testing, causal inference, ML system design, SQL, distributed training, production monitoring & drift, responsible AI.
Open Part 4SQL, NoSQL, vector & graph databases with production depth: data modeling, indexes & query planners, transactions & isolation, replication, sharding, CAP/PACELC, PostgreSQL, Redis/DynamoDB, MongoDB, Cassandra, LSM vs B-tree, the lakehouse, Elasticsearch, vector DBs, graph DBs, Kafka/CDC, datastore selection, DB operations. Two production scenarios per chapter.
Open Part 5Capacity math & SLOs, online/batch/streaming serving, the inference gateway, KServe/Triton/vLLM, feature stores & training-serving skew, retrieval at scale, caching for AI, GPU autoscaling & backpressure, multi-region & fallback models, cost/perf tradeoffs, plus two end-to-end design walkthroughs.
Open Part 6Containers & reproducibility, model/artifact registries, CI for ML, GitOps CD (Argo CD / Flux), progressive delivery (canary, blue-green, shadow traffic), Kubernetes for model workloads, managed inference, eval-in-CI release gates, rollback & kill switches, load testing, migrations, IaC, supply-chain security.
Open Part 7OpenTelemetry, Prometheus & Grafana, golden signals / RED / USE, SLOs & error budgets, LLM telemetry (tokens, cost/request, TTFT, cache-hit), tracing an agent request, eval-in-prod, drift & data quality, guardrail & safety metrics, dashboards, burn-rate alerting, AI-enriched ops, and cost observability / FinOps.
Open Part 8Platform engineering & the paved road, multi-tenancy & quotas, model & data governance, security for AI systems, compliance (EU AI Act, NIST AI RMF, ISO/IEC 42001, SOC 2), responsible AI, FinOps at scale, reliability & cell architecture, LLMOps, org design, vendor portability, a reference architecture, and a readiness checklist.
Open Part 9Anthropic's introduction to working with Claude — prompting fundamentals and core concepts.
Go to course ↗Anthropic's course on collaborating with Claude on real work.
Go to course ↗