The paper evaluates embedding retrieval when surface form and meaning are deliberately separated, testing in two unrelated domains: competition mathematics (500 queries, 117k items) and embodied-agent trajectories (118 queries, 336 trajectories). In mathematics, strict Hit@1 at the heaviest disguise is 0.0%; the correct answer nearly always appears in the top 10 but is outranked by a lexically similar decoy (95.2–99.8% of misses). In trajectories, retrieval performs at or below hypergeometric chance once the gold item involves a different object or receptacle.
A cheap lexical reranker flips sign between domains: it damages mathematical retrieval but improves trajectory retrieval (closing 26–36% of the recoverable gap), providing a diagnostic for whether a benchmark's surface variation is adversarial or incidental. LLM-based rerankers recover 5–76% of the gap but effect sizes and which judge performs best vary with domain, and part of the mathematics gains stem from memorization of well-known competitions. A downstream experiment shows that oracle retrieval and adversarially bad retrieval are indistinguishable (McNemar p = 0.678) because the solver's 69.5% accuracy is largely a truncation proxy (97–100% on finished answers), leaving retrieval no headroom to act.