The authors constructed a corpus of realistic synthetic medical records using Synthea and inserted duplicate canaries (repeated sensitive records) to probe for memorization.
Privacy-aware rewriting offers best privacy-capability trade-off for LLMs on sensitive data
In a controlled study on synthetic medical records, the authors show that rewriting with a uniqueness-aware instruction outperforms direct finetuning and DP-SGD in balancing downstream model capability and privacy leakage.
Big Tech
Zihao Zhao · Avi Schwarzschild · Casey Meehan · Katherine Lee · Zhili Feng · Dylan Sam
Johns Hopkins University · OpenAI
Research Digest··2 min read
The authors conduct a controlled comparison of methods for training language models on sensitive synthetic medical records.
§