TRICKYwalkthrough
Online Schema Changes (Ghost Tables)
Adding a column to a 1-billion-row table cannot lock it. A naive ALTER TABLE acquires a metadata lock, blocking all reads and writes for minutes or hours.
Vitess (PlanetScale) invented this for YouTube's 300K MySQL instances. The control plane coordinates four phases: create a shadow table with the new schema, backfill existing rows in batches of 1,000, capture changes made to the original table during backfill via a change log (binlog), and atomic swap the old and new tables.
“We chose the ghost table approach (not direct ALTER TABLE) because it enables zero-downtime DDL.”
With 1M ranges, serial application takes 1M x 1ms = 17 minutes. Parallelized across 100 workers: 10K ranges per worker at 1ms each = 10 seconds per worker, completing in under 30 seconds total.
Trade-off: the backfill phase doubles storage temporarily and consumes disk I/O bandwidth.
Related concepts