Efficiency & systems

Quantisation, inference serving, kernels, distributed training, compression, efficient architectures

34 articles

All Agents Language models Reinforcement learning Vision Generative media Speech & audio Robotics & embodied AI Safety & security Evaluation & benchmarks Efficiency & systems Code & software AI for science & health Theory

Viewgrid list compact

Pruning experts by replaceability, not magnitude, preserves reasoning in MoE models

Training-free inference framework trims redundant computation and memory in looped Transformers

Mentored decoding can trade small distribution shifts for better predictions

Sparse feed-forward activations cut language-model inference costs substantially

Masked co-adaptive fine-tuning cuts expert fetches by over 23% for MoE models.

Compressed LLM memories can preserve skills beyond their source documents

Restoring softmax gradient symmetry prevents late BF16 attention instability

EntroPack compresses neural weights at finely adjustable storage rates

Approximate scores recover context lost by sparse long-context attention

Input-conditioned fine-tuning preserves language model behavior on unrelated tasks

Simple clustering preserves video understanding with far fewer visual tokens

Simple LRU rivals sophisticated eviction for LLM prefix caching

Training-free feature folding cuts memory and improves accuracy for wide-table tabular models.

Exact quantiles and gradient corrections improve expert load balancing

Training-free MoE expert pruning via consensus residuals preserves reasoning performance

Looped Transformer inference sped up by exploiting sparse cross-loop redundancy

Query-side adapters recover full-context quality in KV cache reuse

LLM-generated PTX rivals autotuned Triton across three GPU generations

Elastic phase sharing improves efficiency in agentic LLM serving

KV cache offloading works when host memory fits agent reuse

Adaptive prefill-decode execution and elastic GPU reuse boost agentic RL throughput

Streaming KV caches makes compacted agent training up to five times faster

Read-only LoRA coupling preserves old skills while adding new ones

Agent workloads expose blind spots in standard LLM inference benchmarks

Prioritizing action-relevant context cuts LLM agent cache costs

DynBranch starts and reuses agent work before branches resolve

Action-aware cache compression makes long-running LLM agents more efficient

Dispatch-aware agents safely optimize compiler-generated GPU kernels within full models

Engineering agents need evidence-bound authorization before their outputs trigger action

Selective attention calls, guided by model's own state, speed long-context inference

Similarity-aware context windows improve model routing across multi-turn conversations

A Vestigial Attention Branch Signals Which Cache Entries to Keep

Full-context drafters help compressed agent models retain accuracy

Cost-aware model routing matches exhaustive estimation with fewer expensive checks