Cloudflare reclaims 100 TB of RAM by optimizing consistent hashing algorithm

Probability analysis allows engineers to cut hash points by 90% while preserving load balancing

By LineZotpaper AI staff

Published24 October 2026

Read Time2 min

Cloudflare engineers have freed more than 100 terabytes of memory from the company's load-balancing service by optimizing the Rust implementation of its consistent hashing algorithm. The changes reduced the number of hash points per backend server by about 90% while maintaining accurate load distribution, according to a blog post from the team.

Cloudflare uses consistent hashing in its Pingora Backend Router to distribute cacheable requests across servers. Supporting weighted servers and different feature combinations required many hash rings, which consumed significant memory.

Kevin Guthrie, principal distributed systems engineer at Cloudflare, Mariia Iurchenko, senior software engineer, Zaidoon Abd Al Hadi, systems engineer, and Ivan Babrou, performance engineer, described in a company blog post how they targeted the pingora-ketama Rust library. They applied probability theory to determine the minimum number of hash points needed.

"Each step down in error margin requires (almost) an order of magnitude increase in the number of hashes per server, so adding more hashes yields less and less improvement," they wrote. The final 90,000 hashes contributed only a 0.7% improvement in distribution error, so the team reduced the total number by 90%.

The team also optimized the Rust data structures, packing hash and server index information into 6 bytes instead of 8, further reducing memory usage.

However, changing the hash ring would redirect cached content, potentially causing a surge in origin traffic. "There was one more problem: changing the hash ring changes where some cacheable requests go. Even if the new ring is better, switching the whole network at once would effectively invalidate almost all cached content. It would turn a memory optimization into an apocalyptic increase in origin traffic," they explained. The new implementation was rolled out gradually alongside the existing ring to limit cache churn.

The work attracted attention online. Hacker News user Teanut commented, "It's fun seeing what little changes at massive scales can do. Good use of math to justify the changes, too."

Cloudflare has undertaken other memory optimization efforts. In August, the company described how its DNS team reduced the memory footprint of its cache through a series of data-structure optimizations.

Analysis

Why This Matters

Reclaiming 100 TB of RAM cuts operational costs and increases capacity for one of the internet's largest content delivery networks.
The application of probability theory to determine the minimum number of hash points provides a blueprint for other large-scale distributed systems.
The staged deployment highlights the practical challenges of changing systems that depend on cache state, where even beneficial changes risk major disruption.

Background

Consistent hashing is a technique used in distributed systems to distribute requests across servers with minimal reshuffling when servers are added or removed. Cloudflare's Pingora Backend Router uses multiple hash rings to handle weighted server configurations and feature variations. The accumulation of these rings led to excessive memory consumption. The team focused on the pingora-ketama Rust library to reduce the footprint.

What to Watch

Further updates from Cloudflare on related memory optimizations in its DNS infrastructure.
Whether other large-scale caching systems adopt the probability-based hash point sizing approach.
The effect of the gradual rollout on Cloudflare's cache hit rates and origin traffic levels.

Sources

Cloudflare Reclaims 100 TB of RAM by Optimizing Consistent Hashing · InfoQ

Checked

12 claims in this story checked against 1 source: 11 supported.

Show each claim

SupportedCloudflare reclaimed 100 TB of RAM by optimizing its consistent-hashing algorithm.InfoQ: “Cloudflare Reclaims 100 TB of RAM by Optimizing Consistent Hashing.”
SupportedProbability analysis let engineers reduce hash points by about 90% while preserving load-balancing accuracy.InfoQ: “The changes compacted hash-ring entries and used statistical analysis of hash distribution and collision rates to reduce the number of hashes per backend by about 90% while preserving load-balancing accuracy.”
SupportedCloudflare reduced the memory footprint of its internal load-balancing service by more than 100 TB by optimizing the Rust implementation of consistent hashing.InfoQ: “Cloudflare recently documented how it reduced the memory footprint of its internal load-balancing service by more than 100 TB by optimizing the Rust implementation of consistent hashing in pingora-ketama.”
SupportedCloudflare uses consistent hashing in its Pingora Backend Router to distribute cacheable requests across servers.InfoQ: “Cloudflare relies on consistent hashing in its Pingora Backend Router to distribute cacheable requests across servers”
SupportedSupporting weighted servers and different feature combinations required many hash rings, increasing memory use.InfoQ: “supporting weighted servers and different feature combinations required many hash rings, significantly increasing memory usage.”
SupportedThe engineers wrote, “Each step down in error margin requires (almost) an order of magnitude increase in the number of hashes per server, so adding more hashes yields less and less improvement.”InfoQ: “Each step down in error margin requires (almost) an order of magnitude increase in the number of hashes per server, so adding more hashes yields less and less improvement.”
SupportedThe final 90,000 hashes improved distribution error by only 0.7%.InfoQ: “the final 90000 hashes improving the error by only 0.7%”
Partly supportedThe team reduced the total number of hashes by 90%.InfoQ: “the team used the analysis to reduce the number of hashes per server by 90%.”
SupportedThe Rust data structures packed hash and server-index information into 6 bytes instead of 8.InfoQ: “the existing representation used an 8-byte Point, even though the hash and server index could be represented in just 6 bytes”
SupportedThe engineers said changing the hash ring could invalidate almost all cached content and cause an “apocalyptic increase in origin traffic.”InfoQ: “There was one more problem: changing the hash ring changes where some cacheable requests go. Even if the new ring is better, switching the whole network at once would effectively invalidate almost all cached content. It would turn a memory optimization into an apocalyptic increase in origin traffic.”
SupportedThe new implementation was rolled out gradually alongside the existing ring to limit cache churn.InfoQ: “The new implementation was rolled out gradually alongside the existing ring to limit cache churn and origin traffic spikes.”
SupportedHacker News user Teanut wrote, “It's fun seeing what little changes at massive scales can do. Good use of math to justify the changes, too.”InfoQ: “User Teanut writes : It's fun seeing what little changes at massive scales can do. Good use of math to justify the changes, too.”