Cloud Storage System Design Walkthrough
Complete design walkthrough with animated diagrams, capacity math, API design, schema, and failure modes.
Solution PathTarget: 30 min
Cloud file storage for 100M DAU syncing 100 billion files across 500M users via block-level delta sync with 4 MB chunks and SHA-256 deduplication saving 40-60% storage, long-polling notification service for sub-5-second sync latency, MySQL metadata sharded by user_id across 16 shards, and fork-based conflict resolution that never loses data.
1/10
1.
What is Cloud Storage?
Dropbox syncs 600 billion content blocks across 700 million users. Google Drive stores over 2 trillion files for 1 billion monthly users.
Drop a file on one device, it appears on every other within seconds. The real challenge is a four-way tension: sync speed, storage cost, bandwidth efficiency, and data correctness.
We need to chunk files into 4 MB blocks so we transfer only what changed, deduplicate identical blocks across users to save 40-60% of storage, sync changes to all linked devices in under 5 seconds via long polling, and resolve conflicts when two people edit the same file offline. These constraints pull in different directions: faster sync means more notification overhead, better dedup means a larger centralized chunk index, and stronger conflict resolution means more version metadata per file.
Dropbox: 600B blocks, 700M users. Google Drive: 2T files, 1B monthly users. Four-stage pipeline: chunk, deduplicate, sync, resolve conflicts.
Dropbox syncs 600 billion content blocks across 700 million registered users. Google Drive stores over 2 trillion files for 1 billion monthly users. These services let users drop a file into a folder on one device, and within seconds it appears on every other device. The system sounds straightforward, but the real challenge is a four-way tension: we must detect which 4 MB blocks changed (chunking), transfer only those blocks (delta sync), deduplicate identical content across all users to save 40-60% of storage (dedup), and resolve conflicts when two people edit the same file offline (conflict resolution).
- Block-level sync transfers only changed 4 MB chunks, not entire files, reducing bandwidth by 99% on typical edits
- SHA-256 deduplication saves 40-60% of storage across all users, worth millions in S3 costs at petabyte scale
- Long polling (not WebSockets) notifies devices of changes in under 1 second with no connection affinity
- Offline edits queue locally and sync on reconnect; version vectors detect conflicts so we never silently discard work