Working notes as I go from RL fundamentals to the LLM-RL frontier (RLHF → DPO → GRPO → verifiable rewards). Written for myself, in public — corrections welcome.
Background: 5+ years production ML, LLM fine-tuning (LoRA/DPO), quantization (GPTQ/GGUF), eval-suite design. Co-author of arXiv:2505.17066 (jailbreak defense, IEEE Access 2025).
| Phase | Material | Status |
|---|---|---|
| 1 — Intuition | David Silver L1–L3, Sutton & Barto ch. 3 | in progress |
| 2 — Core methods | Silver L4–L7, S&B ch. 6, tabular Q-learning impl | — |
| 3 — Deep RL / PPO | CS285 policy gradients, PPO paper | — |
| 4 — RL for LLMs | InstructGPT, DPO, Constitutional AI, GRPO, RLVR | — |
Notes live in notes/, one file per lecture/paper, newest first.