A homelab builder on Reddit has run Moonshot AI's 2.8-trillion-parameter Kimi K3 model on a 40-core third-generation Xeon Scalable CPU and second-generation Intel Optane memory, with no GPU for inference, reporting a sluggish 0.85 tokens per second even with 1-bit quantization.
The build, posted to r/homelab, is a follow-up to an earlier project that used first-generation Optane, and runs Kimi K3 in App Direct Mode, which exposes the memory as a persistent region the software maps directly, rather than Memory Mode, where DRAM acts as a cache. The first build's creator, u/APFrisco, said App Direct can give “more control over where the model's tensors live.”
Keeping the system alive is the hard part. The builder says to “treat active DIMM cooling as mandatory,” running dual 12W blower fans aimed at the modules, which get “extremely hot” with sustained inference; the builder says this can endanger neighboring DRAM. Intel rates the modules for forced airflow of 2.4 to 2.7 metres per second, the kind of cooling found in data centers, and each stick is rated at 12W or 15W regardless of capacity, so a four-stick build could put up to 60W of heat next to the system memory. The Optane throttles at 83 degrees Celsius and trips off at 85, compared with typical DDR4 server memory rated for 85 degrees and up to 95 in extended range.
With a higher 3-bit quantization, the speed drops further to 0.75 tokens per second, and both figures are pre-tuning, before thread counts or NUMA affinity are adjusted. One commenter pointed out that a 50,000-token task was estimated to take about 13 hours at roughly 1 token per second; another noted that overnight batches don't care as much about speed. By comparison, the first build reached roughly 4 tokens per second running the smaller 1-trillion-parameter Kimi K2.5 with 2-bit quantization and an RTX 3060 handling part of the model.
Moonshot AI recommends data center inference engines for Kimi K3, and even the smallest community quantization needs at least 610GB of RAM. With DRAM prices having skyrocketed, the builder says the secondhand Optane cost “a fraction” of the equivalent. Intel launched the Optane PMem 200 series in 2020 for third-generation Xeon Scalable CPUs in 128GB, 256GB and 512GB capacities; on two-socket Xeon platforms, it can deliver an average of up to 32% more memory bandwidth than the first generation.
The builder plans to add GPUs and load parts of the model into DRAM as a faster tier alongside the Optane, with "more benchmarks to come."
Analysis
Why This Matters
- It shows a hobbyist path to running a frontier-scale model at home without a GPU cluster, using surplus server memory once destined for data centers.
- With DRAM prices high, secondhand Optane offers one of the few affordable routes to the 610GB-plus working set Kimi K3 requires.
- The heat and throttling limits underline how far home inference still is from practical, interactive use.
Background
Intel's Optane persistent memory, introduced with the 200 Series in 2020, sat between DRAM and storage, surviving reboots while running far faster than SSDs. Intel has since wound down the Optane product line, leaving secondhand modules available cheaply. Moonshot AI, the Chinese company behind the Kimi models, targets its largest models at data center hardware.
Key Perspectives
The homelab builder: says to "treat active DIMM cooling as mandatory" for sustained inference, and plans to add GPUs and a faster DRAM tier.
Moonshot AI: recommends running Kimi K3 on data center inference engines, not home hardware.
u/APFrisco, creator of the first Optane build: said App Direct Mode gives "more control over where the model's tensors live."
A Reddit commenter: calculated a 50,000-token task would take about 13 hours at roughly 1 token per second.
Another commenter: said overnight batches don't care as much about speed.
What to Watch
- The builder's follow-up benchmarks after adding GPUs and a DRAM tier alongside the Optane.
- Whether the system can stay below the 83-degree throttling threshold during longer runs.
- Whether surplus Optane becomes a common homelab route for giant models while DRAM prices stay elevated.
Checked
12 claims in this story checked against 1 source: 12 supported.
Show each claim
- SupportedA homelab builder ran Moonshot AI’s 2.8-trillion-parameter Kimi K3 on a 40-core third-generation Xeon Scalable CPU with second-generation Optane memory and no GPU for inference.Latest from Tom's Hardware: “A homelab builder on Reddit has run Moonshot AI’s 2.8-trillion-parameter Kimi K3 AI model on discontinued second-gen Optane PMem 200 memory, using a 40-core 3rd-gen Xeon Scalable CPU and not a GPU for inference.”
- SupportedThe build runs at 0.85 tokens per second with 1-bit quantization.Latest from Tom's Hardware: “The project, a follow-up to a separate user’s first-gen Optane build earlier this year, runs at a sluggish 0.85 tokens per second even with 1-bit quantization.”
- SupportedThe builder advised readers to “treat active DIMM cooling as mandatory,” and used dual 12W blower fans aimed at the modules.Latest from Tom's Hardware: “the builder’s advice is that you “treat active DIMM cooling as mandatory,” with dual 12W blower fans aimed at the modules to get temperatures under control.”
- SupportedThe build uses App Direct Mode, which exposes Optane as a separate persistent region that software maps directly; Memory Mode instead uses DRAM as a cache.Latest from Tom's Hardware: “Memory Mode has the Optane act as system memory with DRAM as its cache, the method used in the first build, while App Direct exposes Optane as a separate, persistent region that the software maps directly, the mode used in the second.”
- SupportedThe first build’s creator, u/APFrisco, said App Direct can give “more control over where the model’s tensors live.”Latest from Tom's Hardware: “This can give “more control over where the model’s tensors live,” according to the first build’s creator, u/APFrisco, who planned to try it.”
- SupportedThe builder said sustained inference made the Optane DIMMs “extremely hot” and could endanger neighboring DRAM.Latest from Tom's Hardware: “the Optane DIMMs getting “extremely hot” with sustained inference, creating a situation the builder says can endanger neighboring DRAM.”
- SupportedThe builder said the secondhand Optane cost “a fraction” of the equivalent, amid soaring DRAM prices.Latest from Tom's Hardware: “It doesn’t help that DRAM prices have skyrocketed, and the builder says the Optane memory costs “a fraction” of the equivalent.”
- SupportedAt 3-bit quantization, speed falls to 0.75 tokens per second; the reported speeds are pre-tuning, before thread counts or NUMA affinity are adjusted.Latest from Tom's Hardware: “with the higher 3-bit quantization, the speed went down even more to 0.75 tokens per second.”
- SupportedThe first build reached roughly 4 tokens per second using a smaller 1-trillion-parameter K2.5 model, 2-bit quantization and an RTX 3060 handling part of the model.Latest from Tom's Hardware: “the first build’s roughly 4 tokens per second, although in that case the builder had an RTX 3060 handling part of the model in addition to Memory Mode, a 2-bit quantization, and the smaller, earlier 1-trillion-parameter K2.5 model.”
- SupportedIntel specifies forced airflow of 2.4 to 2.7 metres per second for the modules; each is rated at 12W or 15W, so four sticks could put up to 60W of heat beside system memory.Latest from Tom's Hardware: “the manufacturer rates the modules with forced airflow of 2.4 to 2.7 meters per second for cooling”
- SupportedThe Optane throttles at 83°C and shuts down at 85°C, compared with typical DDR4 server memory rated for 85°C and up to 95°C in extended range.Latest from Tom's Hardware: “Throttling on these occurs at 83 degrees Celsius, with a thermal trip shutdown at 85 degrees, compared to the 85 degrees Celsius and up to 95 degrees in an extended range that typical DDR4 server memory is rated for.”
- SupportedThe builder plans to add GPUs and use DRAM as a faster tier alongside Optane, with “more benchmarks to come.”Latest from Tom's Hardware: “The builder plans to add GPUs and to also load parts of the model into DRAM as a faster tier alongside the Optane, with “more benchmarks to come.””