Cloudflare Cuts 100 TB from 1.1.1.1 DNS Cache with Rust Optimisations

Redesign of Big Pineapple platform reduces per-entry memory footprint by 56% and boosts throughput

edit
By LineZotpaper
Published
Read Time2 min
Cloudflare has freed roughly 100 terabytes of working-set memory across its global fleet by redesigning the in-memory representation of the DNS cache used by its 1.1.1.1 resolver, reducing the per-entry footprint by 56% while also improving lookup latency and insertion throughput.

The changes, applied to Cloudflare's Big Pineapple DNS platform—which stores more than 250 billion cache entries at any given time—involved five successive modifications to the Rust-based cache representation. According to a blog post published by the company, the optimisation increased cache insertion throughput by 43% and reduced lookup latency by 19%.

Systems engineer Sebastiaan Neuteboom described the result on LinkedIn as: "It's not every day you get to save 100 terabytes of memory."

The team replaced Vec and String with Box<[T]> and Box<str> for data fixed after insertion, saving 64 bytes per entry and more than 15 TB fleetwide. They then combined answer, authority, and additional records into a single list with compact offsets, packed Booleans into bitflags, and omitted owner names matching the queried domain, reconstructing them from the cache key.

The largest challenge involved Rust enums. Cloudflare initially boxed larger variants, but separate allocations added overhead and reduced memory locality. The final design stores record data in a contiguous byte buffer using DNS wire format, removing the enum and per-record allocation overhead while improving locality. Frequently used record types can be copied directly into responses, though records containing domain names still require parsing for DNS name compression.

One commenter on Reddit noted that these tricks pay off mainly at Cloudflare's scale: "A lot of these memory tricks only pay off once you're at Cloudflare's request volume; at a smaller scale the extra indirection from boxing variants can actually hurt cache locality more than it helps."

§

Analysis

Why This Matters

  • The 100 TB memory saving directly reduces Cloudflare's operational costs and power consumption across its global network.
  • Faster DNS lookups improve web browsing performance for the millions of users relying on 1.1.1.1 as their resolver.
  • The engineering approach demonstrates how careful data structure design in Rust can yield outsized returns at hyperscale.

Background

Cloudflare operates one of the world's largest public DNS resolvers. Its 1.1.1.1 service handles billions of queries daily, making cache efficiency critical. Big Pineapple is the custom platform underlying this service. The company frequently publishes technical deep-dives on its engineering blog, sharing optimisations that are often adapted by the wider systems programming community.

Key Perspectives

Cloudflare engineering team: Rewriting the cache representation yielded substantial memory and performance gains without requiring new hardware. The tradeoffs of boxing enum variants were carefully managed through a wire-format approach. Smaller-scale operators: The same techniques may not be beneficial at lower traffic volumes; the extra indirection from boxing can degrade cache locality when the working set is smaller. Systems programming community: The work showcases advanced Rust memory management techniques and has sparked discussion on Reddit and other forums about practical tradeoffs in high-performance network services.

What to Watch

  • Whether other large DNS providers adopt similar wire-format cache representations.
  • Cloudflare's future blog posts on further optimisations to Big Pineapple.
  • The impact of these changes on end-to-end DNS resolution times as measured by independent monitors.

Sources

newspaper

Zotpaper

Articles published under the Zotpaper byline are synthesized from multiple source publications by our AI editor and reviewed by our editorial process. Each story combines reporting from credible outlets to give readers a balanced, comprehensive view.