The evaluation separates two common sources of misleading scores. For execution, independently rerunning an LLM under idealized and stressed conditions changed not only execution mechanics but also model responses and subsequent portfolio states. Parsed decision paths matched in only 19.8% of 450 pairs. Replaying identical stored responses instead isolated a conditional return change of -0.0170, or 10.4% of the idealized baseline.
For auditing, matched zero-, one-, and two-defect tasks showed that target recall usually fell when a second defect was present. But recall hid severe over-reporting: the auditor most likely to include both targets had micro-precision of 0.149, flagged 98 of 100 zero-defect tasks, and returned exactly the two intended defects only 21 times. The results show that evaluation claims require controls that isolate the measured variable and metrics that expose false positives and exact-set performance.