Summary: Stanford researchers have developed MUSK, an AI model that integrates visual and language data to improve cancer prognosis and treatment guidance, outperforming conventional methods in accuracy and versatility.
Key Takeaways
- Enhanced Accuracy and Prognostic Capability: MUSK improves cancer prognosis and treatment guidance by integrating multimodal data, achieving higher accuracy than conventional methods—for instance, 77% in identifying lung cancer patients benefiting from immunotherapy versus 61% with standard techniques.
- Use of Unpaired Multimodal Data: The model leverages unpaired visual and language-based data, expanding training datasets without the need for meticulous labeling, enhancing its versatility and scalability for diverse clinical tasks.
- Significant Real-World Impact: Validated on data from The Cancer Genome Atlas, MUSK outperformed standard methods in predicting disease-specific survival, demonstrating its potential to improve personalized cancer care and treatment decisions.
—————————————————————————————————————————————————————————–
Stanford Medicine researchers have introduced an artificial intelligence (AI) model named MUSK (multimodal transformer with unified mask modeling) that combines visual and language-based data for cancer care. MUSK surpasses conventional models in predicting cancer outcomes and guiding treatment strategies.
Transforming AI Use in Clinical Settings
“MUSK can accurately predict the prognoses of people with many different kinds and stages of cancer,” says Ruijiang Li, MD, associate professor of radiation oncology at Stanford Medicine and senior author of the study, published in Nature. “We designed MUSK because physicians never rely on just one type of data to make clinical decisions. By leveraging multiple types of data, we gain more precise predictions about patient outcomes.”
The model was trained on 50 million medical images and over 1 billion pathology-related texts. It predicts cancer prognoses, identifies patients with lung or gastroesophageal cancers who may benefit from immunotherapy, and detects melanoma patients at higher risk of relapse.
Meeting Clinical Needs
Unlike traditional AI tools focused on diagnostics, MUSK addresses the need for prognosis and treatment guidance. “The biggest unmet clinical need is for models that guide patient treatment,” Li says. “Currently, decisions rely on disease staging and biomarkers like specific proteins, but these approaches are often limited in accuracy.”
For example, MUSK was 77% accurate in identifying lung cancer patients likely to benefit from immunotherapy, compared to 61% for standard methods based on PD-L1 protein expression. It also identified melanoma patients at risk of relapse with 83% accuracy, 12% higher than existing models.
A Foundation Model for Medicine
MUSK’s use of unpaired multimodal data expands training datasets without the need for meticulously labeled and paired data. This allows for significantly larger datasets, improving the model’s versatility and precision.
“MUSK’s ability to incorporate unpaired data substantially increases the scale of data available for pretraining,” Li says. “For all clinical prediction tasks, models that integrate multiple data types consistently outperform those relying on imaging or text alone.”
Real-World Impact on Patient Outcomes
To validate MUSK’s capabilities, researchers trained it on data from The Cancer Genome Atlas, covering 16 major cancer types. MUSK achieved 75% accuracy in predicting disease-specific survival, outperforming standard methods based on clinical risk factors, which had 64% accuracy.
By processing data points like tissue imaging, patient demographics, and medical history, MUSK provides detailed insights into treatment effectiveness and prognosis.
Move Toward Personalized Medicine
“The integration of multimodal data with AI models like MUSK is a major advance in improving patient care,” Li says. The team envisions MUSK as a tool that can be customized to address specific clinical questions, helping physicians make more informed treatment decisions.
