Research

New AI papers from arXiv, picked daily and explained with their claims, benchmark results and code.

2566 articles · page 47 of 54 · The Research front page

Evaluation & benchmarks

Small-Sample LLM Rankings Often Overstate Evaluation Reproducibility

Sarkar repeatedly tested eight open model variants on prompt-structure inference, preserving 293 intermediate representations and auditing both output consistency and the resulting rankings. The models frequently produced different structures for identical calls, while bootstrap resampling and alternative merge rules showed that the evaluation identified the worst performers more reliably than the best.

Agents

Scientific agents gain auditability through enforced, re-verifiable research workflows

The IdeaHorizon Team describes AfS, a platform for scientific projects spanning dozens of agent runs with only occasional human involvement. Rather than relying on model instructions alone, AfS mechanically enforces commitments, evidence handling, review boundaries, and memory rules. The paper illustrates these mechanisms through failure traces and two completed campaigns, but does not provide a comparative benchmark.

Agents

Preference learning improves collaboration among teams of language-model agents

Liu et al. formulate preference learning for multi-agent language-model systems and introduce MAPL, which iteratively improves agents by comparing their joint solutions with alternatives from comparator models. Across writing, coding, tool-use and travel-planning tasks, MAPL improved collaboration quality and efficiency, with its learned-reward variant generally outperforming direct preference optimization.

Evaluation & benchmarks

Semantic alignment improves edge classification across unseen dynamic graph domains

Bonnet and Rei introduce a transfer-learning protocol for classifying future interactions in dynamic text-attributed graphs from previously unseen domains. Their experiments show that evaluated self-supervised graph methods underperform a simple text-only baseline on average, while their Spatio-Temporal Semantic Alignment model surpasses both that baseline and the existing methods.

AI for science & health

Incomplete inputs impose unavoidable limits on blind radiomap prediction

Li et al. develop a population-level theory for predicting radio coverage maps from incomplete descriptions of environments and base stations, without field measurements. They show that deterministic models optimally target the conditional mean radiomap, and report that their RadioDecomp framework improves two different propagation-prior baselines across random, unseen-configuration, and unseen-environment tests.

Reinforcement learning

Anchored hypergraphs provide stable credit assignment for multi-agent teams

The authors introduce HySTAR, a MAPPO-based framework that uses an anchored overlapping sparse hypergraph as a temporally consistent value-decomposition basis, while adapting representations via a spatiotemporal encoder. This decoupling prevents structural target drift—the inconsistency in credit assignment when agent interactions change—and yields consistent improvements over MAPPO, value-factorization, and dynamic-grouping baselines across SMAC, GRF, Traffic Junction, and MPE environments. On the hardest SMAC settings, HySTAR achieves relative gains of 16.7% over MAPPO and 15.6% over HYGMA.

Agents

Multi-agent LLM scaling benefits depend on task type and aggregation method

Fortuna and Bertalanič introduce Steiner's taxonomy of group tasks to analyze multi-agent LLM scaling, modeling independently sampled agents as conditionally independent given an item. Testing 13 open-weight models on teams of up to 30 agents across disjunctive and compensatory benchmarks, they find that on disjunctive tasks, plurality voting captures almost none of the potential from larger teams (model predicts within 0.5 points of true accuracy), while multi-round revision yields strong gains that saturate with just one peer. On compensatory Fermi estimation, averaging reduces error by only about 6% because item-level biases shared across samples account for about 87% of squared error.

Vision

End-to-end model writes 3D scene graphs directly from point clouds or Gaussian splats

Milivojevic et al. present GraphWrit3R, a method that takes a 3D point cloud, Gaussian Splats, or a combination as input and directly outputs a complete scene graph as a structured JSON script. On the 3DSSG benchmark, the method achieves state-of-the-art performance on object class, predicate, and triplet recall, outperforming methods that rely on ground-truth object annotations during inference.

Safety & security

Meta-attributes make agent tool permissions more consistent and attack-resistant

The authors developed MetaPermit, an access-control framework that represents the relationship between user intent, execution context and proposed actions through a compact set of meta-attributes. Across two agent benchmarks, it made authorization decisions 31% more consistent than direct LLM authorization and executed no malicious tool calls under five tested indirect prompt-injection attacks.