Video Streaming System Design Walkthrough
Complete design walkthrough with animated diagrams, capacity math, API design, schema, and failure modes.
Solution PathTarget: 30 min
We designed a video streaming platform serving 800M DAU at 46K views/sec with a 460:1 read/write ratio. DAG-based transcoding pipeline splits uploads into 2-second chunks across 8 resolutions for parallel FFmpeg processing (2,400 tasks per 10-min video, under 5 minutes vs 30 minutes monolithic). CDN-first distribution with 95%+ cache hit ratio and request coalescing for viral videos. Resumable uploads, chunk-level failure recovery, and dual-write view counting via Redis plus Kafka batch flush to sharded MySQL.
1/10
1.
What is Video Streaming?
We are designing a platform where a creator uploads a video and within minutes it becomes streamable in multiple resolutions on any device worldwide. YouTube serves 1 billion hours of video per day to 2 billion monthly users. Netflix streams to 230 million subscribers across 190 countries. The real challenge is transcoding 500 hours of video every minute into 8 resolutions while keeping upload-to-playable under 10 minutes.
The core pipeline has four stages: ingest raw video via resumable protocols (files are multi-gigabyte), transcode into multiple resolutions so every device and bandwidth tier gets the right rendition, distribute segments globally via CDN (origin cannot handle 46K concurrent streams), and stream adaptively so the player switches quality based on available bandwidth. Every decision flows from the tension between upload volume, transcoding latency, storage cost, and playback quality.
YouTube: 1B hours/day, 2B monthly users. Netflix: 230M subscribers. Four-stage pipeline: ingest, transcode, distribute via CDN, stream adaptively. The real challenge: transcoding 500 hrs/min into 8 resolutions while keeping upload-to-playable under 10 minutes.
YouTube serves over 1 billion hours of video per day to 2 billion monthly users. A creator uploads a 4K video and expects it to be streamable in 240p through 4K within minutes. The system sounds simple, but the real challenge is turning a single uploaded file into 8 resolution variants, distributing them across 100+ global edge locations, and adapting playback quality in real time for every viewer's bandwidth. We must handle resumable uploads for multi-gigabyte files, run a DAG-based transcoding pipeline that splits video into chunks and encodes them in parallel across 8 resolutions, and deliver content through a CDN with 95%+ cache hit ratio serving 46K views per second.
- Our upload pipeline uses a resumable chunked protocol (5-10 MB chunks with byte offset tracking) so a 10 GB file survives network interruptions. Once uploaded, a DAG-based transcoding pipeline splits the video into GOP-aligned segments, fans them out to parallel FFmpeg workers via a message queue, and encodes each segment into 8 resolutions. A 60-minute video that would take 4 hours on one machine finishes in under 5 minutes across 360 workers.
- We use Adaptive Bitrate (ABR) streaming with HLS or DASH to let the player switch quality every 2-10 seconds based on measured bandwidth. The server generates a manifest file listing all available renditions (240p at 300 Kbps through 4K at 20 Mbps). The player picks the highest rendition that fits within the viewer's connection, avoiding rebuffering on slow networks and maximizing quality on fast ones.
- We deploy a CDN with 100+ Points of Presence (POPs) to cache video segments at the edge, achieving a 95%+ cache hit ratio. Popular videos use push-based pre-warming. Long-tail videos use origin pull with staggered TTLs. At 46K views/sec, the CDN absorbs requests/sec, leaving only 2,300 origin fetches/sec.
- Video metadata (~10 KB per video) lives in MySQL sharded by video_id, cached in Redis. Thumbnails (5 per video, 5 KB each) sit in Bigtable for sub-10ms random reads. View counts flow through Kafka and batch-update MySQL every 30 seconds, with Redis providing real-time counters. Storage grows at 25 GB/sec for raw uploads and 2.16 PB/day across all resolutions.