A team examined explanations generated by a large language model (LLM) for its performance of a clinical prediction task. They had found that, after fine-tuning, the LLM, GPT-4o from San Francisco-based OpenAI, performed comparably to four more traditional types of machine learning (ML) for predicting which patients would discontinue their home cancer medications before planned treatment completion.

From the Department of Biomedical Informatics, research fellow Congning Ni, PhD, Associate Professor Zhijun Yin, PhD, and colleagues reported their findings in the e-book series “Studies in Health Technology and Informatics.” The team used electronic health records and pharmacy surveys from 2,364 cancer patients.

The LLM achieved an F1 score of 87%, while the closest ML model scored 83%. For interpreting the latter model, the team used SHAP, or Shapley additive explanations, a widely used method for exploring (among other things) the internal structure of ML models, revealing how features are weighed. For interpreting the much larger GPT-4o, they asked the LLM to explain its reasoning for individual predictions; to derive feature-importance scores from this output, they used a new method they call mimic-SHAP.

The two models were found to agree on top features — body mass index and age. For secondary features, the LLM was found to lean more on patients’ prior conditions, the ML model on drug exposures and health care procedures.

Many cancer patients discontinue medications taken at home early, for reasons ranging from side effects and lack of response to nonmedical issues such as costs. Predicting early discontinuation could aid efforts to improve treatment adherence.

Others on the study from Vanderbilt include Qingyuan Song, Qingxia Chen, PhD, Lijun Song, PhD, S. Trent Rosenbloom, MD, MPH, Autumn Zuckerman, PharmD, Bridget Lynch, PharmD, MS, and Bradley Malin, PhD. They were joined by Jeremy Warner, MD, MS, of Brown University. The study was supported by National Institutes of Health award R37CA237452.

The post Team’s prediction task compares GPT-4o with classic machine learning appeared first on Vanderbilt Health News.

Genome-wide association studies (GWAS) have discovered thousands of “spots” in the genome associated with diseases, including cancer, but understanding how genetic changes contribute to disease remains a challenge.

Artificial intelligence deep-learning models, such as Enformer, can predict how DNA changes might affect gene regulation. Because these models are trained on broad datasets, however, they do not capture tissue-specific contexts.

A research team led by Qing Li, PhD, and Xingyi Guo, PhD, at Vanderbilt Health, and Quan Long, PhD, at the University of Calgary, has now developed an AI transfer learning approach to adapt Enformer for breast and prostate cancer. Transfer learning is an AI technique that uses a pretrained model (in this case Enformer) as the starting point for a new task. The researchers retrained Enformer using tissue-specific transcription factor chromatin immunoprecipitation sequencing datasets (275 for breast and 357 for prostate).

With the new models, they computed regulatory scores for millions of GWAS genetic variants and identified those most likely to affect cancer risk. They further linked the genes to cancer risk through transcriptome-wide association study analyses and showed that many of the identified genes are important for cancer cell growth and are potential drug targets.

The study, reported in PLOS Genetics, showed that the transfer learning models outperformed the base model in identifying clinically relevant, disease-associated genes. The approach offers a generalizable framework for tailoring foundation models to disease-relevant contexts.

“Our findings demonstrate how adapting existing models to more disease-relevant data can significantly improve our ability to uncover genes and variants involved in cancer,” the authors stated.

Guo and Li are in the Department of Medicine Division of Epidemiology at Vanderbilt Health. The research was supported in part by a Canada Foundation for Innovation John R. Evans Leaders Fund grant to Long.

The post AI technique improves cancer gene discovery for breast and prostate cancer appeared first on Vanderbilt Health News.