Scaling Flight Search to 2 Million Daily Queries on EKS

Flight search looks simple from the outside: a form, a date picker, a list of results. Underneath it, it's one of the highest-fanout systems an airline runs — every query touches inventory, fares, currency conversion, and rules engines across multiple partner systems, all with a latency budget measured in hundreds of milliseconds.

This is the story of how we scaled that system from a single-region deployment struggling under peak load to one that comfortably handles 2M+ searches a day on EKS, without the team growing anywhere near proportionally.

The bottleneck wasn't compute#

Our first instinct, like most teams under load, was to scale out. More pods, bigger nodes, more replicas. It bought us time, but the cost curve was brutal — we were paying for capacity that sat idle outside of peak windows, and we were still seeing p99 latency spikes during actual peak.

The real bottleneck turned out to be fan-out amplification. A single user search was triggering dozens of downstream calls, many of them sequential, several of them redundant across concurrent requests for the same route and date.

Batching, caching, and killing redundant fan-out#

Three changes accounted for most of the improvement:

1. Request coalescing — identical in-flight searches (same route, date, cabin) were merged into a single downstream call instead of each hitting the fare engine independently.
2. Tiered caching — a short-TTL cache in front of fare lookups absorbed the vast majority of read traffic, backed by Redis with a write-through invalidation path for fare changes.
3. Async fan-out with a hard budget — downstream calls that didn't affect the primary result (ancillary offers, upsells) were moved off the critical path entirely.

async function searchFlights(query: SearchQuery) {
  const cacheKey = buildCacheKey(query);
  const cached = await cache.get(cacheKey);
  if (cached) return cached;
 
  return coalesce(cacheKey, async () => {
    const results = await fareEngine.search(query, { timeoutMs: 350 });
    await cache.set(cacheKey, results, { ttlSeconds: 20 });
    return results;
  });
}

That coalesce wrapper — a simple in-memory promise map keyed by request signature — removed a surprising amount of load on its own. During a fare sale, the same route/date pair might be requested hundreds of times within the same second across different users.

Scaling on EKS without over-provisioning#

With the fan-out problem solved, horizontal scaling on EKS became about matching real demand rather than compensating for waste:

• Horizontal Pod Autoscaler tuned against request latency, not just CPU
• Node groups split by workload shape (bursty search traffic vs. steady background jobs) so autoscaling decisions didn't fight each other
• Circuit breakers around every downstream dependency, with degraded responses (cached or partial results) instead of hard failures

The result: the same regional footprint now serves more than 2 million searches a day, with steadier latency at peak than we had at a fraction of that volume before the changes.

The actual lesson#

None of this required exotic infrastructure. It required treating fan-out as a first-class scaling concern, not an implementation detail hidden behind a service boundary. The biggest wins came from questioning why a request was making ten downstream calls in the first place — not from adding more pods to absorb the cost of those calls.