TRICKYwalkthrough
Exactly-Once Counting: What It Actually Takes
"We use Kafka, so we have exactly-once" is the checkbox answer, and interviewers set traps for it. Walk what a duplicate actually costs: a stream worker crashes after adding click #4,000,001 to the counter but before committing its offset; the replacement replays and adds it again: the counter is now wrong forever, and if that counter bills, money is wrong forever.
Link one: idempotent event identity. Every click carries a click_id minted at capture (a Snowflake ID from topic 17): so "the same click twice" is detectable at all.
“Exactly-once counting requires an unbroken chain of three guarantees, and breaking any link silently downgrades the whole pipeline to at-least-once.”
Link two: transactional state and offsets. The processor's counter state and its input offsets must move atomically: Flink does this with checkpoint barriers (state snapshots aligned with offset positions: on crash, both rewind together, so replayed events re-produce identical state rather than double-counting); Kafka Streams does it with transactions spanning offset commits and output produces.
Link three, the one everyone forgets: the sink. If the processor emits minute-aggregates to a database non-transactionally, a crash between emit and checkpoint duplicates the output even though the state was perfect: sinks must be idempotent (upsert keyed by (ad, window): our choice, because re-writing the same aggregate is naturally safe) or transactional (two-phase commit with the checkpoint).
The honest engineering posture: end-to-end exactly-once is achievable inside the stream system's boundary and at cooperative sinks: and impossible past uncooperative ones (a webhook, an email): which is precisely why the billing path re-derives from the raw log instead of trusting any stream-side promise. What if the interviewer asks: is the dedup-by-click_id table not enough by itself?
At 500K clicks/sec, a lookup per event doubles the hot-path cost: dedup-by-identity is the batch path's exhaustive tool; the stream path leans on checkpoint atomicity and reserves identity checks for the capture edge (double-fired browser events).
Related concepts