Search

6 results for “WikiText-2”

1. Research · Efficiency & systems

   Mixed-precision LLM accelerator turns lower bit weights into proportionally faster compute

   FIGFLEX: A Mixed-Precision FP-INT GEMM Accelerator Co-Designed with Low-Bit LLM Quantization

   A radix-4 bit-parallel datapath and GPTQ-aware quantizer let FIGFLEX match or beat GPU-level accuracy at 2 to 3 bit precision while raising throughput as precision falls.

   WikiText-2: 5.85 perplexity (as reported)

   23 Oct 2026University of California, Irvine

2. Research · Efficiency & systems

   Curvature-aware pruning preserves influence functions efficiently.

   Two For The Price Of One: Curvature-Aware Pruning Preserves Influence Functions Too

   CACTUS, an ADMM-based optimizer, achieves order-of-magnitude speedups over prior methods while retaining gradient alignment and enabling data selection.

   WikiText-2: 14.17 perplexity (as reported)

   14 Oct 2026Massachusetts Institute of Technology, Google Research

3. Research · Efficiency & systems

   Lower reconstruction loss can degrade quantized LLM performance; robust refinement fixes it

   When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantization

   DRQ refines integer codes to minimize worst-case loss over activation distribution shifts, improving six PTQ methods without inference overhead.

   WikiText-2: 9.2 perplexity (as reported)

   10 Oct 2026University of Science and Technology of China, University College London, Shanghai University of Finance and Economics

4. Research · Efficiency & systems

   H-SVDQuant improves 4-bit quantization of diffusion transformers with low-rank branches

   Deflating the Hessian: Rank-4 W4A4 Quantization for Multimodal Diffusion Transformers

   A unified framework jointly modeling weight and activation quantization error achieves better performance with rank 4 than existing methods with rank 32.

   WikiText-2: 10.44 perplexity (as reported)

   10 Oct 2026The University of Hong Kong, Huawei Technologies Co., Ltd.

5. Research · Efficiency & systems

   Canonical training improves language models under joint low-bit compression

   Few Bits, One Law: Toward W2A4KV2

   CanonQ uses fixed transforms and reusable codebooks, then jointly trains models to tolerate quantized weights, activations and attention caches.

   WikiText-2: 14.86 perplexity (as reported)

   8 Oct 2026Meta AI

6. Research · Efficiency & systems

   Combining model compression techniques can collapse small language models

   Compression Technique Interactions in Small Language Models: A Factorial Ablation Study

   A systematic study of 72 compression configurations shows that INT8 quantization is lossless, while NF4 with pruning causes total failure, and LoRA fine-tuning recovers most quality without a teacher model.

   WikiText-2: 97.4% perplexity reduction after LoRA recovery (as reported)

   3 Nov 2026Amazon Web Services