The authors curated and transcribed 12,534 minutes of real multidisciplinary tumor board meetings from YouTube, resulting in 611 patient cases and 19,157 discussion turns involving ten specialist roles.
LLMs struggle to match multidisciplinary tumor board discussions in new benchmark
OpenTumorBoard evaluates 14 models on 611 patient cases, finding best scores of only 3.43 out of 5 for clinical equivalence and 2.78 out of 5 for board consensus alignment.
Big Tech
Anqi Li · Zhixuan Ge · Yixuan Duan · Jiarong Qian · Chi-Yu Chen · MingYu Lu · +6 more
Rice University · University of Illinois at Urbana-Champaign · University of Washington · National Yang Ming Chiao Tung University · Microsoft
Research Digest··3 min read
The authors introduce OpenTumorBoard, a benchmark derived from 12,534 minutes of publicly available tumor board recordings on YouTube, comprising 611 patient cases and 19,157 discussion turns across ten specialist roles.
Why this paper
From Microsoft and 5 others
In one line
OpenTumorBoard benchmarks LLMs on multidisciplinary tumor board discussions with 611 cases, revealing maximum scores of 3.43/5 and 2.78/5.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§