As AI agents gain the ability to execute code, call APIs, and interact with external systems with minimal human oversight, two research teams have published independent frameworks aimed at making those agents safer — one by automatically refining safety rules over time, the other by placing an inviolable enforcement layer outside the agent's own control.
Autonomous AI agents — systems that combine large language models with access to external tools, databases, and computing environments — are becoming increasingly capable of completing complex tasks without step-by-step human guidance. That growing autonomy, however, brings a growing set of risks: agents may execute destructive commands, exfiltrate sensitive data, or violate domain-specific constraints, often without any immediate human awareness.
Two papers published this week on arXiv propose distinct but complementary approaches to the problem.
AutoSpec: Teaching Safety Rules to Improve Themselves
Researchers from several institutions introduced AutoSpec, a framework designed to automatically evolve safety rules for AI agents. The system begins with expert-designed rules and then refines them continuously as it encounters real-world examples that the original rules handle incorrectly.
The core challenge AutoSpec addresses is a persistent tension in AI safety engineering: hand-crafted rules are easy to understand and audit, but they age poorly. Rules that are too conservative block legitimate operations — known as false positives — while rules that are too permissive allow unsafe actions to slip through.
AutoSpec tackles this by combining counterexample-guided inductive synthesis (CEGIS) with inductive logic programming (ILP). The system examines annotated execution traces, identifies where existing rules fail, and uses ILP to pinpoint which logical predicates best distinguish safe from unsafe behaviour. It then proposes and verifies rule edits before accepting them.
Tested on 291 execution traces across code execution and embodied agent domains, AutoSpec achieved F1 scores of 0.98 and 0.93 respectively — a measure combining precision and recall — while reducing false positives by up to 94 percent. The system converged within four to five iterations, and the resulting rules remained human-readable and auditable.
The Unfireable Safety Kernel: A Lockbox Outside the Agent's Reach
A separate team — Seth Dobrin and Łukasz Chmiel — took a different angle, arguing that any safety control placed inside an AI agent's own runtime is fundamentally vulnerable. Their paper introduces the concept of "escapable AI systems": any AI with sufficient reach into its own execution environment can, in principle, be manipulated into bypassing controls embedded within that environment.
Their solution, the Unfireable Safety Kernel, is implemented in Rust and operates as a separate process from the agent itself. All actions the agent wishes to take must pass through this kernel before execution — a structurally enforced chokepoint the agent cannot circumvent or disable, hence the name.
The kernel satisfies four properties the authors argue are necessary for genuine architectural control: process separation, pre-action enforcement, fail-closed behaviour at multiple levels, and externally verifiable signed evidence of decisions.
Formal verification underpins the design. The kernel's fail-closed invariant was confirmed using an SMT theorem prover (Z3) and an exhaustive bounded-model-checking proof. In empirical testing against a self-improving AI system used as an adversary, all 704 attempts to modify the safety-critical core were refused across 1,000 self-modification attempts, with no successful escapes. A separate campaign of 6,240 authorisation round-trips also produced no bypasses.
Different Approaches, Shared Urgency
The two papers address different layers of the same problem. AutoSpec focuses on making the content of safety rules more accurate and adaptable. The Unfireable Safety Kernel focuses on ensuring that whatever rules exist are enforced in a way the agent cannot undermine. Neither paper claims to solve AI safety comprehensively, but both frame their work as addressing gaps left by existing approaches — including training-time alignment techniques such as reinforcement learning from human feedback (RLHF) and Constitutional AI, which operate before deployment rather than during it.
Analysis
Why This Matters
- AI agents are already being deployed in enterprise and consumer settings with access to real systems — email, databases, code interpreters, and financial APIs. The gap between their capabilities and the maturity of safety infrastructure around them is a live risk, not a theoretical one.
- Both papers highlight a structural weakness in current AI safety thinking: most alignment work happens during model training, but agents can behave unsafely at runtime in ways training cannot fully anticipate. Runtime enforcement is an underexplored layer.
- The AutoSpec approach — producing auditable, human-readable rules — speaks directly to emerging regulatory demands in the EU AI Act and similar frameworks that require explainability and auditability in high-risk AI systems.
Background
The deployment of AI agents accelerated sharply from 2023 onward, as companies began integrating large language models with tool-use capabilities — allowing them to browse the web, write and execute code, send emails, and interact with APIs. Frameworks such as LangChain, AutoGPT, and later OpenAI's function-calling interface made agent construction accessible to a wide range of developers.
Early safety efforts focused almost entirely on training-time interventions: RLHF, Constitutional AI, and fine-tuning on curated datasets. These techniques shape model behaviour in general but cannot anticipate every context an agent might encounter once deployed. Output filters and system-prompt guardrails emerged as a secondary layer, but security researchers have demonstrated repeatedly that prompt injection attacks and adversarial inputs can circumvent them.
The formal study of runtime or execution-time alignment for agents remains relatively nascent. A handful of academic groups and AI safety organisations — including Anthropic, DeepMind, and various university labs — have published work on sandboxing, capability control, and interpretability, but no widely adopted standard for agent safety architecture has emerged.
Key Perspectives
AI Safety Researchers: The two papers represent progress toward a more layered, defence-in-depth model of AI safety — one that does not rely solely on training or on trusting the agent to police itself. The formal verification underpinning the Unfireable Safety Kernel is particularly notable, as machine-checked proofs are rare in applied AI systems work.
Developers and Operators: Practical adoption will depend on integration overhead. The AutoSpec framework requires annotated traces, which means organisations must invest in labelling infrastructure. The Unfireable Safety Kernel's Rust implementation adds an architectural layer that existing Python-based agent pipelines may not easily accommodate.
Critics and Sceptics: Neither paper has yet undergone independent peer review or real-world deployment at scale. The adversarial testing described — while rigorous by academic standards — involves controlled scenarios. Critics may argue that sufficiently capable future agents could find attack surfaces not anticipated by current kernel designs, and that the arms race between agent capability and safety enforcement is far from resolved.
What to Watch
- Whether either framework is adopted or cited by major AI platform providers (OpenAI, Anthropic, Google DeepMind) as a basis for production agent safety infrastructure.
- Regulatory timelines: the EU AI Act's provisions for high-risk AI systems begin applying in 2026, which may create commercial demand for auditable, runtime-enforced safety mechanisms of exactly the kind these papers describe.
- Follow-up empirical testing by independent researchers attempting to bypass the Unfireable Safety Kernel — the adversarial robustness of formally verified systems is a known area of ongoing debate in computer security.