Concepts

A standard set of tasks or data used to evaluate and compare the performance of AI systems.

Checking outputs for correctness before accepting them, often to catch errors or reward hacking.

A neural network trained on massive text corpora to generate coherent text and perform diverse language tasks.

Measuring AI agents' performance in complex tasks, highlighting failures in planning, monitoring, and real

A machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.

A model that handles tasks combining visual and textual information, such as editing images from conversation or reasoning across images and text.

A repository where an agent stores past interactions and outcomes to inform future decisions.

A type of generative model that learns to reverse a gradual noising process to create data.

Determining which actions or steps in a sequence contributed to a final outcome, often for reinforcement learning.

Risks from AI agents including timing errors, shutdown avoidance, collusion, overclaiming, and unsafe execution, mitigated by constrained decoding, safety harnesses, and swarm governance.

A reinforcement learning method that optimizes policy by computing advantages based on group-level reward statistics.

A system that structures, verifies, monitors, or controls how an AI agent interacts with external tools, environments, or execution runs.

A generative training method that learns an ordinary differential equation mapping noise to data by matching velocity fields along probability paths.

Reinforcement learning that uses rewards from objectively checkable signals like correct answers or verification feedback.

An AI-powered software system that autonomously writes, modifies, or extends source code based on high-level instructions.

Identifying which component or step in a system caused an observed failure, often using structured trace representations.

The study of how neural networks compute through internal circuits, features, and causal interventions, rather than just input-output behavior.

Audits an AI agent's live actions, internal states, or effects against constraints or policies to catch errors and attacks.

A training method where a student model learns from a teacher using its own current policy's outputs, often improving reasoning and instruction following.

Training approach that lets LLM-based agents learn multi-step tool use and planning via reinforcement learning, often with dense or per-tool rewards.

Learned capabilities that enable AI agents to perform specialized tasks with improved effectiveness.

Allocating extra computation during inference to improve output quality.

A predictive environment model that agents use to simulate outcomes, enabling planning, reasoning, and long-horizon task execution.

A system where multiple AI agents collaborate, each handling specialized roles, communicating and coordinating to complete complex tasks.

Comparing actual outcomes with imagined alternatives to estimate the effects of different actions or conditions.

A systematic process for verifying data integrity, model behavior, and evaluation validity through checks for errors, confounds, and hidden artifacts.

Pulling specific, relevant information from a large corpus to support an answer, decision, or reasoning step.

Process of training models to recognize and refuse harmful requests, ensuring outputs remain safe.

An agent that improves by reusing its own experience, memory, or self-generated feedback, with success depending on verification and design.

The collaborative operation of multiple AI agents to solve tasks more effectively or with greater capability.

Confidence calibration aligns a model's predicted confidence with its actual accuracy across predictions.

Adapting a deployed model during inference using new data or interactions, without a separate training phase.

A neural architecture with many specialized sub-networks, only a few activated per input via a router, balancing capacity and efficiency.

A robot policy model that maps visual observations and language instructions to actions, trained on demonstrations and often used for manipulation tasks.

A security attack where malicious instructions are embedded into inputs to override an LLM's intended behavior.

Information available only during training that improves performance but is not accessible at inference.

Directing each query or subtask to the most suitable model to optimize cost, accuracy, or quality.

Creating training examples artificially, often using an LLM, to overcome real-data scarcity or privacy constraints.

Compressing or summarizing an LLM's past context into a smaller representation to extend memory and improve reasoning within limited token budgets.

Reducing the size of the key-value cache in transformers via selective retention, low-rank updates, or adaptive compression.

Activation steering modifies a model's internal vector representations during inference to shift behavior, such as editing outputs, removing capabilities, or controlling trust.

A granular assessment of a system's intermediate steps or subgoals, rather than just its final output, to diagnose specific failures.

A neural network architecture combining transformer layers with diffusion processes for high-quality image and video generation.

A learning paradigm where a model sequentially acquires knowledge from multiple tasks without significantly forgetting earlier ones.

Reasoning precisely about object positions, orientations, and spatial relations across planar, depth, and temporal dimensions for robotics and navigation.

Reward hacking occurs when an AI agent exploits loopholes to achieve high reward without meeting the intended objective.

A post-training method that adjusts model behavior using comparisons or self-generated preferences to improve alignment.

Iteratively adapting an agent's environment, prompts, or evaluation setup to improve task performance and generalization.

An AI system repeatedly uses its own outputs to enhance its capabilities, creating a feedback loop for autonomous improvement.

A paradigm where models learn representations from unlabeled data by predicting hidden or transformed parts of the input.

A method that uses a lightweight model to propose token sequences that are then verified in parallel by a larger model.

A learned internal, compressed representation of environment dynamics, enabling future-aware planning, generation, and reasoning directly in latent space.

Quantizing pre-trained neural network weights and activations to

Selectively attending to only the most relevant positions or tokens to reduce computational overhead.

The capability of an agent to invoke external tools sequentially over multiple conversational turns to complete complex tasks.

In policy optimization, a method that calculates the relative value of actions using group statistics or baselines to guide learning.

Separating distinct functions such as planning, reasoning, and execution into separate components improves efficiency, quality, and reliability.

An AI agent that interacts with computer interfaces—GUI, terminal, or code—to autonomously perform tasks.

A method that learns embeddings by minimizing distance between positive pairs and maximizing distance between negative pairs.

An evaluation suite that measures agent performance on tasks spanning entire codebases, from construction to repair and governance.

The process of developing, refining, or optimizing reusable capabilities in LLM agents through learning, adaptation, or experience accumulation.

Feedback that evolves alongside the policy or components it guides to improve performance.

Summarizing or structuring past information to reduce token usage while preserving task accuracy.

Autonomous agents automatically generate, control, or extend simulations to improve accuracy, interactivity, or performance.

External supports like training curricula, tool selection, and success verifiers enable agents to handle long-horizon or out-of-distribution tasks.

Evaluation that assesses model performance by considering the real-world impact or risk of errors, not just standard accuracy.

The ability of a system to preserve and recall information over time or across contexts, often balancing compression against fidelity.

Same inputs give different outputs across runs, confounding testing and evaluation.

A predictable power-law relationship between compute or data and model performance.

Having access to relevant evidence does not prevent an LLM agent from making incorrect or unsafe decisions.

A paradigm where actions are expressed as executable code, such as method calls or program synthesis, rather than direct manipulation.

Using demonstrations or solutions from one task, domain, or agent to improve performance or training in another.

A single end-to-end model processes and generates multiple data types (e.g., images, text) with equal competence.

An encoder predicts the latent representation of future data points from current and past observations, bypassing pixel-level reconstruction.

A language model that generates text by iteratively denoising a latent or discrete representation.

A hidden trigger in a model causes malicious behavior only when activated, while the model otherwise appears benign.

A training method that aligns student and teacher output distributions to compress many-step generative models into efficient few-step versions.

A transformer architecture whose weights are reused across repeated passes, trading loop count for depth, compute, or accuracy.

A training strategy that gradually increases task difficulty based on the learner's current performance, often using success rates to select the next challenge.

Tracking state with explicit data types, separated from

An attention mechanism that computes attention with linear, rather than quadratic, complexity in sequence length.

A multi-step AI system that autonomously gathers, evaluates, and synthesizes evidence from diverse sources to produce comprehensive reports.

Mapping natural language commands or descriptions to specific visual elements of a user interface.

Training a neural network with simulated quantization in the forward pass so its weights and activations adapt to low-precision formats, improving deployment accuracy.

Systematic skew in how an agent samples actions or evidence, often trapping it in unproductive regions or distorting policy learning.

Multi-agent systems collaborate autonomously to produce long-form videos maintaining narrative and visual coherence.

A formal privacy guarantee that limits how much an algorithm’s output reveals about any single individual’s data.

Multiple autonomous agents coordinate and combine their specialized capabilities to solve complex tasks beyond individual ability.

Combining different attention types (e.g., linear and full) or positional/semantic strategies in one model to balance efficiency, accuracy, and memory.

Planning that uses extra situational information to guide decision-making and improve outcomes.

A multi-step process that generates or verifies spatial arrangements of objects against explicit constraints and geometry.

A method for updating the probability of a hypothesis based on new evidence.

Removing specific knowledge, behaviors, or concepts from a trained model while preserving overall capability.

Latent communication transfers information between models via internal states like KV caches instead of explicit text.

A structured web of entities and relations, used to store domain facts and support reliable reasoning, retrieval, and model alignment.

Vector quantization encodes data as indices into a fixed set of codebook vectors, reducing storage and computation at some cost in precision.

Training a model on adversarially perturbed inputs to make it robust to attacks, often trading some clean accuracy for resilience.

A neural network that learns a compact, interpretable representation of data by activating only a small subset of its neurons at once.

A learned model generates plans directly for new instances, replacing expensive per-instance search with a fast, generalizable forward pass.

A 3D scene representation using overlapping Gaussian ellipsoids for efficient, high-quality novel-view rendering.

The task of converting spoken audio into text, evaluated across dialects and used for training efficient transcription models.

A learned scoring function that rates outputs to align generation, assess evidence, or optimize inference.

Jointly models world dynamics and action policy to generate rollouts for training.

Loss of previously learned knowledge when a model is fine-tuned on new tasks.

A search algorithm that expands multiple promising candidates simultaneously to balance generation quality and computational cost.

AI models abandoning correct answers to align with a user's incorrect beliefs or hinted choices.

A model that generates realistic, controllable video sequences by learning the underlying physical and temporal dynamics of a scene.

Compounding bias is the accumulation of small errors or biases that worsen over multiple steps or longer sequences.

A statistical framework estimating latent abilities from task responses, enabling adaptive evaluation and comparable ability scores across diverse items.

The modeling error from fitting training data too precisely, causing poor generalization to new data; often countered by regularization.

Associating visual information in images with corresponding language descriptions or abstract concepts.

Adversarial probing of AI systems to uncover security and safety vulnerabilities before real-world deployment.

A method that produces prediction sets with guaranteed coverage, enabling risk control without distributional assumptions.

A method that trains a simple classifier on a model's internal representations to predict a specific property or concept.

A control method that uses a model to predict future outcomes and selects actions to optimize performance over a horizon.

Distributing training compute across several independent model runs during an intermediate phase improves final performance over a single long run.

A pretrained deep learning model for time series forecasting that generalizes across tasks or datasets.

A method of prompting an AI system to bypass its safety guardrails and produce restricted or harmful outputs.

A method that estimates rare event probabilities by sampling from a modified distribution and extrapolating from those samples.

Learning-based technique that makes representations from different models or modalities compatible by mapping them to a shared space.

A mathematical method for finding the cheapest way to move probability mass between distributions, used to align, denoise, or repair data representations.

An AI system used as a testbed to study alignment failures like collusion and scheming.

Generates video frames one at a time, each conditioned on previously generated frames.

Training a language model to predict multiple future tokens at once, rather than one at a time, improving generation speed and efficiency.

Controls data movement across trust boundaries to block injection attacks and data leaks.

Using knowledge gained from one task to improve learning on a different but related task.

The model uses its own predicted expected embedding to condition subsequent denoising steps.

A directed knowledge structure where skills are linked, enabling targeted updates and efficient retrieval for AI agents.

Reasoning carried out in hidden states instead of explicit steps, allowing parallel exploration and faster inference.

A search algorithm that uses random sampling to evaluate actions and builds a tree for planning.

A nested optimization where solving an outer problem requires finding optimal parameters for an inner problem.

Training technique using ground-truth tokens as step-by-step autoregressive input instead of the model's own generated history.

Autoregressive video diffusion generates video frames sequentially, conditioning each new frame on previously generated ones.

Splitting LLM serving into separate prefill and decode phases to optimize GPU resource allocation and batch sizing.

Coordinating a humanoid robot's arms, legs, and torso to perform manipulation and locomotion tasks together while maintaining balance.

A mathematical structure for dynamic systems, with a latent state evolving over time to generate observations or predict future behavior.

A harmful fine-tuning-induced state where models prefer harmful answers on unrelated tasks, stemming from a narrow parameter subspace that projection can suppress.

Quantization that maps groups of weights or values to the nearest point in a regular lattice, like E8, enabling compact codes without a codebook.

Shortcut learning occurs when models exploit spurious correlations or superficial features instead of the intended task-relevant patterns.

Mechanisms by which a model stores and reproduces exact training data, especially sensitive information, rather than learning generalizable patterns.

Reusing the same training samples across epochs, affecting overfitting, model degradation, and scaling-law predictions in pretraining.

A learned scoring function that predicts human preferences to guide reinforcement learning, typically trained on feedback data.

Training large language models to accommodate or reflect multiple, sometimes conflicting, human value perspectives rather than a single dominant one.

Using linear classifiers or decoders to extract features from model hidden states for analysis or task improvement.

Adding new domains or modalities to an existing pre-trained model by training it further on curated data beyond its initial pre-training.

Automatically converting informal instructions into formal specifications for verification and enforcement.

A model that imitates human behavior for training or evaluating conversational agents in realistic interactions.

A delayed phase change where held-out accuracy improves even as training loss remains flat or rises.

Software whose behavior is driven by an embedded AI agent, enabling dynamic adaptation, autonomous task execution, and contract-based integration.

A fast, automatic model that produces quick responses with little deliberation, often used for efficiency at the cost of reasoning accuracy.

Supervision techniques for AI systems that scale to catch rare errors and ensure correctness through protocols like debate.

The practice of scaling model size and data volume to achieve optimal performance for a given compute budget.

A system for monitoring and regulating AI agent behavior to prevent covert or harmful actions during tasks.

Iterative correction of a model's own outputs during generation or selection to improve quality without retraining.

A memory component that persists across many time steps in a neural network, storing information separately from short-term states.

Residual connections in attention mechanisms retain past states for efficient, continuous improvement in transformer models.

Selectively removing or deactivating experts in a mixture-of-experts model to improve efficiency while preserving accuracy.

Training agents to reach specified goals, with rewards and exploration dependent on the goal.

An optimization algorithm for neural network training that rivals AdamW on vision transformers but shows mixed results on language models.

A method that generates multiple output tokens simultaneously in a single forward pass to accelerate inference.

Using visual perception to guide interaction with digital interfaces—clicking, navigating, and executing multi-step actions.

A reinforcement learning method that optimizes policy parameters by following gradients of expected reward from sampled trajectories.