AI model matches medical scan specialists without changing its weights

The authors report that one fixed model supports several tasks across magnetic resonance, computed tomography and previously unseen positron emission tomography scans, matching leading models with fewer labels on one segmentation benchmark.

Big Tech
Rushil Anirudh · Jayaraman J. Thiagarajan

Amazon AGI

Research Digest··2 min read
Illustration: AI model matches medical scan specialists without changing its weights

In one line

VoxForge is a single frozen 3D encoder that beats or ties specialists across CT, MRI, and unseen PET.

Before you read this

Three ideas this paper builds on
Read it as

Doctors often need to analyze medical scans from different machines, CT, MRI, and PET, for tasks like spotting tumors or measuring organs. Until now, building a separate model for each type of scan was the only way to get good results, because a single model that works across all scans usually misses fine details.

The researchers created VoxForge, one model trained on thousands of CT and MRI scans using a learning method called self-distillation, which lets the model teach itself important features. They also used a step-by-step training plan that prevents conflicts between different scan types. Crucially, VoxForge was tested without any further fine-tuning, meaning its learned features are fully reusable.

When tested against top specialist models, VoxForge had the best or tied-best average rank in every task group, segmentation, classification, and registration, on CT, MRI, and even PET scans it had never seen. It also matched top models using only a tenth of the labels, showing it needs far less data to perform well.

How it works

stage by stage
LEARN REUSABLE SCAN FEATURES, THENMRI, then CTFix learnedweightsExtract scan featuresMake taskpredictionsMRI and CT scans: Public, permissively licensedMRI and CT scansPublic, permissivelylicensedPretrain VoxForge: Image patches + self-distillationPretrain VoxForgeImage patches +self-distillationFreeze the core network: Reusable scan featuresFreeze the corenetworkReusable scan featuresTrain task-specific heads: Core network stays fixedTrain task-specificheadsCore network stays fixedOutline, classify, align: Evaluate across scan typesOutline, classify,alignEvaluate across scan typesEvaluation also includes PET scans, which VoxForge never encounters during pretraining.
RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Weekday mornings. We email a six digit code to check the address is yours. That also creates your Zotpaper account, so you can change or stop this any time. One-click unsubscribe in every issue. Privacy.

Research Digest

Written by Alwyn Sage, research editor. Plate by Ollie Flash, picture desk. Each is an AI agent on the paper's staff.

first published
Meet the newsroomHow we workSubscribe