Language model libraries often fail outside their core calculation code

Gao and colleagues attribute 60.33% of 300 sampled library issues to compiler, build-system or runtime-environment causes, rather than the kernel-library category.

Big Tech

Yanjie Gao · Jiayi Bi · Haoxiang Lin · Ling Feng

Microsoft Research Asia · Tsinghua University

Research Digest·9 November 2026·2 min read

Illustration: Language model libraries often fail outside their core calculation code

In one line

GPU kernel library failures often originate in build and runtime environments, while specialized assumptions and inadequate testing allow correctness and hardware-specific problems to slip through.

Before you read this

Three ideas this paper builds on

failure attributionIdentifying which component or step in a system caused an observed failure, often using structured trace representations.132 papers →verificationChecking outputs for correctness before accepting them, often to catch errors or reward hacking.312 papers →runtime monitoringAudits an AI agent's live actions, internal states, or effects against constraints or policies to catch errors and attacks.125 papers →

Read it as

Large language models run on graphics processing units (GPUs), where small, optimized pieces of code called kernels do much of the computation. Reusable kernel libraries help them run quickly, but failures can stop installation, crash a job, or quietly produce wrong answers.

To understand these failures, researchers examined issue reports, discussions, and fixes from several libraries, grouping symptoms, causes, and debugging practices.

Among the reported problems, 60.33% came from tools that translate and package code, or from the software and hardware programs run on. Specialized kernels also depend on assumptions about input shapes, memory arrangements, and hardware. Testing needs to cover how software is built and deployed, not just whether its calculations work.

How it works

stage by stage

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

software engineering processes failure attribution verification runtime monitoring