
LLM Engineering From Scratch
Overview LLM Engineering From Scratch is an expanding public lab. The implemented work currently centers on a byte-level BPE tokenizer with deterministic artifacts, stress cases, failure analysis, explanations, and an interactive merge demo. Later model components remain roadmap items. Roadmap inspiration: Ahmad Osman (@TheAhmadOsman) and his article, “Step-By-Step LLM Engineering Projects (2026 Edition)”. The repository uses the article as a roadmap reference; the implementations, experiments, traces, demos, and writeups are independent. Links GitHub Repository Tokenizer From Scratch Blog Post BPE Merge Microscope Demo Status The first project, Tokenizer From Scratch, is implemented with a byte-level BPE tokenizer, deterministic artifacts, and an interactive static demo. The planned sequence continues through embeddings, positional methods, attention, Transformer blocks, training loops, and objectives. ...