The authors define the prior barrier as the log-ratio of pretrained probabilities for competing concepts versus the target concept.
Prior barriers quantify uneven pretrained concept support for fine-tuning
The authors show that rare concepts require more instructions to overcome higher prior barriers, motivating an adaptive selection method.
Big Tech
Haohui Wang · Jiahao Xu · Wangzhi Zhan · Tong Zeng · Dongqi Fu · Hong Li · +6 more
Virginia Tech · Amazon · Meta · MBZUAI · Dartmouth College
Research Digest··2 min read
The authors introduce 'prior barriers' to measure how strongly a pretrained model favors competing concepts over a target concept.
Why this paper
From Amazon and 4 others
In one line
Prior barriers in pretrained models follow a long-tail distribution, requiring adaptive instruction selection for effective fine-tuning.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§