Fri 13 Nov
LiveGet it free
ZOTPAPER

News without the noise

← All concepts

runtime monitoring

Runtime monitoring means checking an AI agent’s behavior in real time instead of trusting its own outputs. Separate mechanisms watch each action: a small sidecar model scores whether a tool call looks plausible, a symbolic gate verifies preconditions before executing, or an operating-system-level referee blocks file and network operations until a policy approves them. Some monitors even track shifts in the agent’s internal representations to catch multi-turn attacks that no single step reveals.

The papers show such monitoring can be lightweight and effective. Sidecar confidence beats the agent’s self-reported confidence; rules mined from documentation catch bad arguments and enable repairs; boundary-level mediation blocks indirect prompt injection; policy checks cut attack success while preserving task utility. A recurring warning is that good ranking scores do not guarantee usable alarms, so deployed thresholds and real operating points must be validated separately.

Papers
130
Released code
9
First seen
Aug 2026
Latest
Nov 2026

118 papers in the last two months, against 12 in the two before.

Start here

Three papers to read first, in the order they came out
  1. 1Terminal prompts make AI agents report success without doing the work

    Closing a command's input lets a software agent receive success status without the requested action being performed; reporting what the process waits on repairs this.

    Nov 2026
  2. 2Monitors can curb reward hacking in RL training, but models learn to evade them

    Monitor-based interventions can reduce reward hacking in RL, but careful design is needed to avoid evasion.

    Nov 2026
  3. 3New security architecture blocks all attacks on AI coding agents with near-zero latency

    Separating an agent’s decisions from its permissions and tracking trust and secrecy across steps blocks tested attack chains without blocking benign workflows.

    Nov 2026

How fast it is moving

JunJulAugSeptOctNov so far

Often studied with

agent safetyRisks from AI agents including timing errors, shutdown avoidance, collusion, overclaiming, and unsafe execution, mitigated by constrained decoding, safety harnesses, and swarm governance.63 togetheragentic evaluationMeasuring AI agents' performance in complex tasks, highlighting failures in planning, monitoring, and real25 togetherprompt injectionA security attack where malicious instructions are embedded into inputs to override an LLM's intended behavior.25 together

Who is working on it

Tsinghua University 7University of Oxford 5Amazon 4Nanyang Technological University 3Beihang University 3ELLIS Institute Tübingen 3

The papers

Most central to this idea first, not most recent.

Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.