(LLMs) need vast memory for their weights. Accessing this memory during use is slow and consumes much energy. Analog in-memory computing (AIMC) lets computation happen right where weights are stored, reducing this delay. To hold all weights, memory arrays are stacked in 3D (3D-IMC). However, how weights are placed in these stacks determines how many computations can run at once, affecting speed.
CAMP, a software algorithm, analyzes the specific LLM, the chip design, and the expected usage. It builds a conflict matrix that predicts which weight calculations would clash for resources, then refines the placement using a profiling feedback loop.
CAMP increased parallel computation throughput by up to 2.0 times and overall system throughput by up to 1.74 times compared to existing methods. This speeds LLM inference without any hardware changes.