Huawei unveils OceanStor M900 memory storage to tackle AI inference bottleneck

Vendor says context-memory appliance extends KV cache into shared storage and cuts latency for agentic workloads

By LineZotpaper

Published19 October 2026

Read Time3 min

Huawei has introduced OceanStor M900 Context Memory Storage at HUAWEI CONNECT 2026 in Shanghai, a storage platform designed to ease the memory bottleneck facing AI inference as context windows grow and agentic workloads multiply. The company claims the system can deliver up to 64PB of shared cache capacity, cut access latency to 60 microseconds and roughly double token throughput in typical AI coding workloads.

Huawei says the demands of AI inference are changing as organizations move from short interactions towards longer conversations, complex reasoning and autonomous agents. Those workloads place heavy pressure on memory capacity, and keeping accelerators supplied with data is becoming as important as raw computing power.

A particular challenge is the key-value (KV) cache, which stores information generated during inference so models can reuse previous computations. As context windows expand and multi-turn interactions grow more demanding, these caches get considerably larger. GPU memory and system memory offer fast access but are expensive and capacity constrained, while traditional storage provides more capacity but relies on longer, CPU-mediated data paths that can leave accelerators waiting.

OceanStor M900 is designed to bridge that gap. According to Huawei, it adds a dedicated Context Memory Storage layer that extends the KV cache beyond accelerator and system memory into a shared, multi-tier storage pool. Using Huawei's UnifiedBus high-speed interconnect, the platform lets KV cache resources be shared across a cluster. The company claims capacity of up to 64PB and substantially more context available per NPU.

The system's ASU disk integrates CPU, networking and storage controller functions. Combined with native KV semantics, Huawei says this allows NPUs to access stored cache data in a single hop, avoiding conventional protocol conversion and CPU forwarding. The company claims the approach reduces access latency to 60 microseconds, potentially doubling token throughput and halving time to first token in typical AI coding workloads.

Cost and durability are also part of the pitch. Huawei's KV-aware adaptive storage technology manages data placement according to cache lifecycles, with claimed endurance of up to 24 drive writes per day, or 16 times that of conventional SSDs. The company argues this could reduce replacement and maintenance costs over time.

The wider significance, Huawei says, lies in treating storage not as a separate supporting component but as an interconnected part of inference performance. As agentic AI workloads grow, how efficiently infrastructure retains, shares and retrieves context may become as important as how quickly accelerators can process it.

Analysis

Why This Matters

AI inference is shifting from compute-bound to memory-bound work as context windows lengthen, and infrastructure that cannot feed data fast enough leaves expensive accelerators idle.
Huawei is arguing that storage, not just faster chips, will determine inference performance, signalling a broader architectural change for datacenters.
The product targets hyperscale deployments and Huawei's Ascend SuperPoD infrastructure, strengthening its full-stack AI offering.

Background

Huawei has been building out its own AI hardware stack, including its Ascend line of processors, while its OceanStor brand has long been associated with enterprise storage. OceanStor M900 brings those two strands together by treating memory and storage as one shared resource for inference. The idea of moving KV cache data closer to accelerators reflects a wider industry push to reduce the time accelerators spend waiting for context data.

Key Perspectives

Huawei: The company positions OceanStor M900 as a way to make memory, networking and compute tightly interconnected. It claims faster access, higher throughput and longer drive endurance, translating into lower running costs. Customers and analysts: These are launch-day figures from a sponsored vendor announcement. The performance claims, including doubled token throughput and 60 microsecond latency, have yet to be demonstrated in independent, real-world deployments.

What to Watch

Whether third-party benchmarks or early customer deployments confirm the claimed throughput and latency figures.
How broadly OceanStor M900 is adopted beyond Ascend SuperPoD environments.
Whether other infrastructure vendors introduce similar context-memory storage layers in response.

Sources

AI inference is running into a memory bottleneck. Huawei wants to tackle it with OceanStor M900 · www.theregister.com - Articles

Checked

9 claims in this story checked against 1 source: 6 supported.

Show each claim

SupportedHuawei unveiled OceanStor M900 memory storage to tackle the AI inference bottleneck (headline).www.theregister.com - Articles: “AI inference is running into a memory bottleneck. Huawei wants to tackle it with OceanStor M900.”
SupportedThe OceanStor M900 was introduced at HUAWEI CONNECT 2026 in Shanghai.www.theregister.com - Articles: “At HUAWEI CONNECT 2026 in Shanghai, Huawei introduced OceanStor M900 Context Memory Storage”
SupportedThe system can deliver up to 64PB of shared cache capacity, as the company claims.www.theregister.com - Articles: “with Huawei claiming capacity of up to 64PB and substantially more context available per NPU.”
SupportedThe company claims the system cuts access latency to 60 microseconds.www.theregister.com - Articles: “According to Huawei, this approach reduces access latency to 60 microseconds, potentially doubling token throughput and halving time to first token in typical AI coding workloads.”
Partly supportedThe system roughly doubles token throughput in typical AI coding workloads.www.theregister.com - Articles: “According to Huawei, this approach reduces access latency to 60 microseconds, potentially doubling token throughput and halving time to first token in typical AI coding workloads.”
Partly supportedThe system halves time to first token in typical AI coding workloads.www.theregister.com - Articles: “According to Huawei, this approach reduces access latency to 60 microseconds, potentially doubling token throughput and halving time to first token in typical AI coding workloads.”
Partly supportedThe product cuts latency for agentic workloads (summary).www.theregister.com - Articles: “According to Huawei, this approach reduces access latency to 60 microseconds, potentially doubling token throughput and halving time to first token in typical AI coding workloads.”
SupportedThe platform extends KV cache beyond accelerator and system memory into a shared, multi-tier storage pool.www.theregister.com - Articles: “extending KV cache beyond accelerator and system memory into a shared, multi-tier storage pool.”
SupportedHuawei says the platform's single-hop access avoids protocol conversion and CPU forwarding for NPUs.www.theregister.com - Articles: “Combined with native KV semantics, this allows NPUs to access stored cache data in a single hop, avoiding conventional protocol conversion and CPU forwarding.”