Tokenizer From Scratch: BPE as Learned Compression
A from-scratch byte-level BPE tokenizer with runnable Python, failure cases, charts, and an interactive trace that shows every merge step.
Tokenization
A from-scratch byte-level BPE tokenizer with runnable Python, failure cases, charts, and an interactive trace that shows every merge step.