What happens when builders work on real systems

We spend a lot of time optimizing database queries. We check indexes, count page reads, and argue about whether Redis cluster sizes should be upgraded. But latency is a human experience.

If your app takes 300ms to react to a click, it feels broken. If it takes 50ms, it feels like an extension of the user's mind.

The p99 Metric Average latency is a comfort blanket for engineers who want to ignore outliers. The p99 metric—how the worst 1% of your users experience your product—is the only number that tells the truth.

In our Kochi-London datastore sync, we saw averages of 12ms. But the p99 was spiking to 1,200ms when WAN link packets dropped during monsoon weather. We rewrote the network queue routing to use disk-buffered write queues. The average rose slightly to 15ms, but the p99 dropped to a stable 78ms. That is the difference between average data and human experience.