Predicting cancer prognoses and responses to treatment
Estimated reading time: 6 minutes
MUSK outperforms standard methods in predicting the prognosis of cancer patients including their response to treatment options. This AI powered tool, developed by Stanford Medicine, draws from over a billion datasets, moving us a step closer to an era of personalised cancer medicine
Scientists at Stanford Medicine have developed an artificial intelligence tool that accurately predicts prognoses and treatment responses for cancer patients, a new study has found.
The team trained the model using 50 million medical images of standard pathology slides from 11,577 patients, and more than a billion pathology-related texts, enabling it to interpret both visual and language-based information.
Stanford’s team found their AI foundation model outperformed standard methods in its ability to predict the prognoses of thousands of patients with diverse types of cancer. For example, it was able to identify which patients with lung or gastroesophageal cancers would be more likely to benefit from immunotherapy. It was also able to establish the patients with melanoma who were most likely to experience a recurrence of their cancer.
Improving patient care
Researchers named the AI tool MUSK, short for multimodal transformer with unified mask modelling. They say it represents a “marked deviation” in the way AI is currently used in clinical settings. They also believe it could transform cancer patient care.
“MUSK can accurately predict the prognoses of people with many different kinds and stages of cancer,” explains Ruijiang Li, an associate professor of radiation oncology, who is the senior author of the study that was recently published in the journal, Nature.
“We designed MUSK because in clinical practice, physicians never rely on just one type of data to make clinical decisions. We wanted to leverage multiple types of data to gain more insight and get more precise predictions about patient outcomes.”
What’s unique about MUSK, he says, is its ability to incorporate unpaired multimodal data into pretraining as this “substantially increases the scale of data compared with paired data required by other models”.
“We observed that for all clinical prediction tasks, models that integrate multiple types of data consistently outperform those based on imaging or text data alone,” Li adds. “Leveraging these types of unpaired multimodal data with AI models like MUSK will be a major advance in the ability of artificial intelligence to aid physicians to improve patient care.”
MUSK outcomes
To train MUSK to predict disease-specific prognoses, the team collected microscopic slides of tissue sections, their associated pathology reports, and follow-up data (including how the patients fared) from the US national database, the Cancer Genome Atlas.
They considered data for people with 16 major types of cancer, including breast, lung, colorectal, pancreas, kidney, bladder, head and neck. This information was then used to train the AI model to predict disease-specific survival, or the percentage of people who have not died from a specific disease, during a defined time period.
For all cancer types, MUSK accurately predicted the disease-specific survival of a patient 75% of the time. In contrast, standard predictions based on a patient’s cancer stage and other clinical risk factors were correct 64% of the time.
“The biggest unmet clinical need is for models that physicians can use to guide patient treatment,” Li explains. “Does this patient need this drug or should we instead focus on another type of therapy, for example? Currently, physicians use information like disease staging and specific genes or proteins to make these decisions, but that’s not always accurate.”
Assessing immunotherapy responses
MUSK was also trained by the researchers to use thousands of snippets of information to predict which patients with cancers of the lung or of the gastric or oesophageal tract would be most likely to benefit from immunotherapy. Li tells us more.
“At the moment, the major determination about whether to give a patient a particular type of immunotherapy rests on whether that person’s tumour expresses a protein called PD-L1,” he explains. “That’s a biomarker made of just one protein.
“In contrast, if we can use artificial intelligence to assess hundreds of thousands of bits of many types of data, including tissue imaging, as well as patient demographics, medical history, past treatments and laboratory tests gathered from clinical notes, we can much more accurately determine who might benefit.”
With non-small cell lung cancer, MUSK correctly identified patients who benefitted from immunotherapy treatment about 77% of the time. In contrast, the standard method of predicting immunotherapy responses, which is based on PD-L1 expression, was correct only around 61% of the time.
Similarly, when the researchers trained MUSK to identify which people with melanoma would be most likely to relapse within five years after their initial treatment, the model was correct around 83% of the time. This suggests MUSK is at least 12% more accurate than the predictions generated by other AI foundation models.
Potential impact of MUSK
Dr Marianne Baker is science engagement manager at Cancer Research UK and says AI models such as MUSK could help to improve accuracy both with cancer prognosis and in terms of treatment recommendations.
“Artificial intelligence models can help to bring lots of different types of information together to improve accuracy when doctors predict how someone’s cancer is likely to behave,” Baker explains. “They can also identify which treatments might be most effective.
“It’s important that clinicians can use as much information as possible when recommending treatment for people with cancer,” she adds.
“With further testing and development, AI models like MUSK could assist cancer doctors in making accurate prognoses and help them give their patients the best treatment options.”
This AI tool could potentially be useful in other areas of medicine outside of cancer care, says Yang Tang, a healthcare, AI and technology investment expert.
“The MUSK model is more than just an academic milestone. It represents a shift towards truly integrated AI systems in healthcare,” he explains.
“Its applications could extend to radiology, dermatology, and other fields requiring multimodal analysis. In an era where personalised medicine is becoming the gold standard, MUSK lays the groundwork for scalable, accurate, and efficient clinical decision-making.”
Connect with Ruijiang
innovation spotlights the future of RARE through a technological lens. To access more innovation articles click below.
