The authors built SysRB, a benchmark of 934 source-tree pairs from the Linux kernel and projects covered by OSS-Fuzz.
Coding agents often miss crashes and misread changes that prevent them
Tests on 934 paired source trees show that spotting crash-related code does not reliably translate into predicting whether an input will cause a failure.
Big Tech
Chenxi Huang · Haonan Wang · Jinghe Zhang · Aleksandr Nogikh · Franjo Ivančić · Petros Maniatis · +4 more
Columbia University · Google Inc. · Google DeepMind
Research Digest··2 min read
Huang and colleagues tested four coding agents on source-code versions that produce opposite crash outcomes for the same input.
§