Safety & security

Alignment, red teaming, jailbreaks and attacks, guardrails, privacy, interpretability for safety

300 articles · page 1 of 4

All Agents Language models Reinforcement learning Vision Generative media Speech & audio Robotics & embodied AI Safety & security Evaluation & benchmarks Efficiency & systems Code & software AI for science & health Theory

Viewgrid list compact

Filtering atypical records lets differential privacy noise match typical data sensitivity

Training chatbots to state a safety decision before answering defeats multi-turn attacks

New watermark for language models detects AI text without distorting output or slowing decoding

Fake content and structure in tool outputs combine to fool all tested AI agents

AI agents follow attacks in documentation 2.6 times more than in code

Safety training can make chatbots refuse even after they start complying

AI auditors find more problematic behavior by reading models’ internal signals

Malicious skill files can make AI agents answer forbidden questions

One in seventeen AI agent skills contains a directive that bypasses human oversight

Medical AI teams launder demographic bias into triage calls no audit catches

Private-query AI assistants leak secrets through rejected drafts, new protocol plugs the hole

AI agents can inherit malicious commands from poisoned pretraining data

Poisoned tool descriptions can make AI agents leak data after legitimate work

Data privacy tools calculate more accurate totals by considering outliers’ direction

AI forecasting models take the top two spots across three benchmarks

Robots handle visual changes better by reusing previously successful movements

Language models can keep key parts yet systematically misidentify names after compression

A single attack triggers malicious behavior in LLMs across multiple deployment-time transformations

Language models can be steered with ratings, not just yes-or-no labels

AI training’s privacy losses resemble a bell curve when individually small

AI agents can bypass access restrictions using public web services

AI models adopt patients' false diagnoses 23% of the time in medical tasks

Language models’ jailbreak weaknesses depend more on training randomness than data

AI agents take harmful actions on notices they have judged untrustworthy

Chatbot defenses can miss poisoned documents padded with meaningless words

AI code reviewers miss one in six attacks spread across updates

Most language models rarely acknowledge information that influences their answers

Language models’ self-assessments can look accurate without predicting future performance

AI agents resist mistaken criticism better when they record evidence first

AI reviewers reward confident writing even when the science stays unchanged

Individually approved agent skills can combine to break workflow policies

AI agents break chatbot safeguards more often in fewer exchanges

AI image editors can be tricked into copying rather than editing

Language models’ rare safety failures can be estimated with fewer labeled trials

Chatbots follow higher-priority rules better after training on conflicting goals

Image generators forget unwanted content more reliably while preserving image quality

AI agents helping each other can bypass oversight and harm outsiders

Language models favor suggested answers even when told the source is unreliable

Language models’ saved versions can reveal supposedly deleted training text

AI training can be detected in simulated datacentre power despite concealment

Dermatology concept annotations show systematic skin-tone bias that models inherit and detectors miss

Language models’ safety monitors improve predictably with size and labeled examples

AI interpretation tools can miss mixed meanings in their most influential features

Retraining AI monitors on one cover-up reduces other hidden cover-ups

AI email agents can handle standard phishing safely yet obey tailored attacks

AI simulations can predict safeguard effectiveness better than tests in separate environments

Monitoring research agents detects integrity violations and helps reduce them

Persistent memory in LLMs enables cross-session security breaches

Embedding-free framework enables private topic discovery in LLM conversations

Edge vision-language models are as physically attackable as classic object detectors

LLM agents often ignore safety refusals when using tools

AI text detectors can expose readable features without sacrificing performance

Translation bypasses safety alignment in low-resource languages across multiple LLMs

AutoRedClaw automates red-teaming for persistent claw-style agents

Robot safety check catches harmful commands missed by control loop

Polarity blindness causes LLM safety guardrails to miss harmful intent after negation

Frontier models can covertly sabotage their AI successors in recursive self-improvement

Empirical denoiser enables privacy accounting for diffusion sampling

Diffusion model fairness shifts with sampling settings, not just weights.

Deception in LLMs is not a single direction, adaptive subspace probes work better

Malicious subagents can persuade supervisors to exceed authority boundaries

LLM agents exceed granted authorization in nearly one in five tasks

New benchmark shows health chatbots agree with self-diagnosis when they should caution

Orthogonal Boundary Unlearning preserves benign skills while removing dangerous ones

Black-box attack defeats major LLM watermarking schemes while preserving text quality

Activation monitors gain only modest advantage from original model computations

LLM agents decide to follow prompt injections during reasoning, not at injection time

Automated framework discovers hidden biases in vision-language models

Misspecified training rewards reduce the reliability of chain-of-thought monitoring

Debate Corrects Human Judgments, but Usually Fails Weaker AI Judges

Splitting attacks across AI agents sharply weakens per-agent monitoring

Persistent agent state lets prompt injections survive across sessions

LookAhead Defense uses the model's own safe behavior to block dormant poisoning.

Multi-agent LLM safety requires separate evaluations of communication, authorization, and execution

Fine-grained flow controls block browser-agent attacks without restoring origin walls

Many learned transcoder features closely reproduce individual transformer neurons

Some LLM graders collapse jailbreak compliance into evasion, skewing attack success rates

Benign fine-tuning can teach aligned models to answer harmful requests

Sycophancy predates post-training but its specifics form during fine-tuning

Jailbreak success rates flip with evaluator and sampling choices

Mid-training stage internalizes safety without sacrificing general capability

RL Training Can Amplify Compliance That Disappears Without Oversight

Post-training amplifies a shared circuit for following misleading cues

Backdoor attack on federated prompt tuning exploits frozen backbone to evade detection

Narrow harmful training can broadly invert reward model preferences

Attack success rate misses latent harmful continuations reachable after a refusal

AI image generators produce faces judged more attractive than real people and encode racial bias

Privacy-aware rewriting offers best privacy-capability trade-off for LLMs on sensitive data

Recursive self-distillation on benign outputs can progressively misalign safe models

Non-English requests raise tool admission and response harm in LLM agents

Beacon restores safety signals lost during group-relative reasoning alignment

Repository prompt injections can widen coding agents’ lasting permissions

Document training can forge couplings that make fine-tuning elicit unintended behaviors

Sparse coding replaces linear encoders to stabilize SAE concept discovery

Continual learning in deepfake detectors widens demographic unfairness over time

ChatGPT's health answers are high-risk for one in five real users