The changes, applied to Cloudflare's Big Pineapple DNS platform—which stores more than 250 billion cache entries at any given time—involved five successive modifications to the Rust-based cache representation. According to a blog post published by the company, the optimisation increased cache insertion throughput by 43% and reduced lookup latency by 19%.
Systems engineer Sebastiaan Neuteboom described the result on LinkedIn as: "It's not every day you get to save 100 terabytes of memory."
The team replaced Vec and String with Box<[T]> and Box<str> for data fixed after insertion, saving 64 bytes per entry and more than 15 TB fleetwide. They then combined answer, authority, and additional records into a single list with compact offsets, packed Booleans into bitflags, and omitted owner names matching the queried domain, reconstructing them from the cache key.
The largest challenge involved Rust enums. Cloudflare initially boxed larger variants, but separate allocations added overhead and reduced memory locality. The final design stores record data in a contiguous byte buffer using DNS wire format, removing the enum and per-record allocation overhead while improving locality. Frequently used record types can be copied directly into responses, though records containing domain names still require parsing for DNS name compression.
One commenter on Reddit noted that these tricks pay off mainly at Cloudflare's scale: "A lot of these memory tricks only pay off once you're at Cloudflare's request volume; at a smaller scale the extra indirection from boxing variants can actually hurt cache locality more than it helps."