Samsung unveils LPDDR5X-PIM memory with on-chip AI processing at Hot Chips 2026

The South Korean memory giant has been developing PIM technology since 2021, initially targeting high-bandwidth memory (HBM) stacks used in AMD accelerators. Now, Samsung has brought the concept to the more affordable and power-efficient LPDDR5X standard, aiming to address the growing cost and energy demands of AI workloads.

At Hot Chips 2026, Samsung presented the design of its LPDDR5X-PIM, which integrates a small amount of logic — including multiply-accumulate (MAC) units and vector register files — directly into each memory bank. Unlike its HBM-PIM implementation, which required cutting some banks to fit the logic, the new LPDDR5X-PIM adds PIM capability to every bank without sacrificing capacity. The memory can operate in two modes: single-bank (standard DRAM) or multi-bank (PIM), with a traditional DRAM controller switching between them as needed.

To overcome reordering challenges when mixing standard reads/writes with PIM operations, Samsung developed Address Align Mode (AAM), which maps DRAM addresses to MAC instructions. In a typical inference task where weight parameters are already stored in memory, the PIM logic can perform multiplication and accumulation operations directly, bypassing the processor bottleneck.

Samsung's benchmark results show that LPDDR5X-PIM is 2.28x faster than standard LPDDR5X in AI inference, and when combined with bandwidth optimizations, the gain reaches 3.01x. The company also highlighted that LPDDR5X consumes significantly less power than DDR5 or HBM, making it an attractive option for edge AI and mobile devices.

However, the technology faces hurdles. PIM requires software support and memory controllers that can handle the dual-mode operation. Samsung said it expects JEDEC to publish an initial specification for LPDDR6X-PIM later this year, suggesting the industry is moving toward standardization.

The announcement comes amid warnings from Micron about the widening silicon gap between HBM and DDR5, with HBM wafer demand straining supply and costs. LPDDR5X-PIM could offer a middle ground for applications that need higher performance than standard RAM but cannot justify HBM's expense.

Why This Matters

Cost relief for AI systems: Memory already accounts for a large and growing share of AI chip costs. LPDDR5X-PIM offers a cheaper alternative to HBM for inference workloads, potentially lowering the barrier for deploying AI in edge devices, smartphones, and servers.
Power efficiency: LPDDR5X's low power profile combined with PIM could enable always-on AI assistants, real-time video analytics on laptops, and other battery-constrained applications that currently rely on cloud processing.
Standardization momentum: Samsung's push for a JEDEC specification on LPDDR6X-PIM signals a shift from proprietary HBM-PIM to industry-wide standards, which could accelerate adoption across the memory ecosystem.

Background

Samsung first introduced the concept of processing-in-memory in 2021 by modifying AMD MI100 accelerators with HBM-PIM. The idea is simple: move simple computations — especially multiply-accumulate operations common in neural networks — into the memory array, reducing the amount of data that must travel between memory and processor. Early benchmarks showed power savings of up to 80% in some workloads.

HBM-PIM has remained niche because HBM itself is expensive and primarily used in high-end data center accelerators. Meanwhile, the demand for AI inference at the edge has exploded, with applications ranging from on-device language models to real-time image processing. Standard LPDDR5X, while power-efficient, still suffers from the von Neumann bottleneck: data must travel through a narrow memory bus to the CPU or NPU.

Samsung's LPDDR5X-PIM aims to break that bottleneck in a more affordable package. At Hot Chips 2024, the company demonstrated a similar concept for LPDDR5X but said it was still in the concept phase. Now, two years later, the product is validated and ready for customer sampling.

Key Perspectives

Samsung Electronics: Positions LPDDR5X-PIM as a practical solution to the growing memory wall problem. By integrating logic into each bank without reducing capacity, Samsung claims it offers the best of both worlds — high density and local compute. The company emphasizes backward compatibility with existing memory controllers, expecting a software-driven transition.

Micron Technology: A competitor that has focused on HBM and DDR5 rather than PIM. Micron recently warned that the silicon gap between HBM and standard DRAM is widening, implicitly arguing that PIM in lower-power memory may not bridge the performance gap needed for the most demanding AI workloads.

System integrators and AI chip designers: They must decide whether to adopt PIM memory or stick with traditional architectures. PIM simplifies data movement but complicates programming. Startups like Groq and Tenstorrent have pursued alternative near-memory compute approaches, and skepticism remains about whether PIM's performance gains justify the engineering overhead.

Critics: Some memory experts question whether the additional logic die area and design complexity will erode the cost advantage over HBM. Others point out that PIM only speeds up memory-bound operations like matrix multiplication — if the rest of the network (e.g., activation functions, pooling layers) struggles, the overall gain may be limited.

What to Watch

Adoption in mobile SoCs: Watch for flagship smartphone chips from Qualcomm, MediaTek, or Samsung Exynos that support LPDDR5X-PIM in 2027–2028.
JEDEC standardization of LPDDR6X-PIM: Samsung expects an initial spec this year — if it materializes, it will force other DRAM makers (SK Hynix, Micron) to adopt PIM or risk losing compatibility.
Real-world inference benchmarks from independent reviewers: Samsung's 2.28x – 3.01x claims are under controlled conditions; third-party testing on workloads like LLaMA or Stable Diffusion will reveal true gains.

ZOTPAPER