TRICKYwalkthrough
Live Traffic from GPS Probes
It is 5:30 PM on a Friday. A three-car accident closes two lanes on Highway 101 in Silicon Valley.
How does the system detect this without any human reporting it? The answer is GPS probe data from millions of phones.
“Within 60 seconds, Google Maps shows the highway segment in red.”
Roughly 20 million users are actively navigating at any moment, each sending a GPS update every 5 seconds. That is .
Each update contains latitude (8B), longitude (8B), speed (4B), heading (4B), timestamp (8B), and device metadata (8B), totaling about 40 bytes. Throughput into the ingestion layer: .
We pipe this through Kafka partitioned by geographic tile. But raw GPS coordinates have 5 to 10 meter error in open sky and 50 meters or more in urban canyons.
A GPS point might land on a parallel frontage road instead of the highway. We solve this with map matching: a Hidden Markov Model (HMM) where the hidden states are road segments and the observations are GPS readings.
The Viterbi algorithm finds the most likely sequence of road segments the driver actually traversed. Once matched, we compute each road segment's current speed as the median speed of all probes on that segment within a 60-second sliding window.
We track 50 million road segments globally. Each segment stores a current speed (4B), sample count (2B), confidence score (4B), and timestamp (8B), totaling 18 bytes.
Total in Redis: . For segments with fewer than 5 probes in the window, we apply confidence weighting: the reported speed is a weighted average of the sparse real-time data and the historical average for that segment, time of day, and day of week.
This prevents a single slow-moving delivery truck from painting an entire highway segment red. Trade-off: a shorter window (30 seconds) detects incidents faster but amplifies noise from individual drivers.
A longer window (5 minutes) is smoother but delays detection. Google and Waze settled on roughly 60 seconds as the sweet spot.
What if the interviewer asks: what about privacy? Speed data is aggregated and anonymized.
No individual trajectories are stored after map matching. Segments with fewer than 5 unique probes are suppressed entirely.
Related concepts