Blog

A ground-up, engineer-friendly guide to the ideas behind Transformers — the sequence problem, why RNNs failed, and the tokenization, embeddings, and positional encoding that set the stage for attention.

A from-scratch, jargon-busted guide to self-attention — Query/Key/Value, scaled dot-product, a fully worked numerical example, multi-head, causal masking, and why long context is expensive.

A hands-on tour of the original encoder-decoder Transformer: Add & Norm, feed-forward networks, cross-attention, the output head, and how the whole thing is trained.

Encoder-only, decoder-only, encoder-decoder — meet the three Transformer families behind BERT, GPT, and T5, and learn exactly why decoder-only models took over.

A deep, engineer-friendly survey of the upgrades — RoPE, GQA, FlashAttention, Pre-LN, RMSNorm, SwiGLU, MoE, and scaling laws — that turned the 2017 Transformer into a modern LLM.

How a raw transformer becomes a helpful assistant like ChatGPT or Claude — the full pretraining, fine-tuning, and RLHF pipeline, plus how decoding actually turns probabilities into text.