Machine Learning for Clinical Risk Prediction in Big Data
Keywords:
Combining PC-1; PC-2 with ML Ridge regression; SVM Linear; lasso; AUC-ROC curve; area under the precision-recall; maximal accuracy Ya.Yu. et al.; the goodness of fit was evaluated and an internal; external validation of the risk score Risk factors; sensitive; specific; especially; AUC-ROC 3829 clinical/biobanked; 110706 clinically well-characterized; 7400 clinical; 165237 individuals.Abstract
Healthcare is a path-breaking field for big data. By combining electronic medical record data with omics data (genomics, proteomics, metabolomics, etc.), lifestyle information (e.g., smoking, drinking, diet, and exercise), social determinants of health, and relevant data from wearable devices, a diverse array of clinical and biological predictive models can be constructed. In particular, the application of machine-learning (ML) methods for clinical risk-prediction modeling has gained impressive momentum in recent years, amassing a wealth of reference literature. Unlike traditional statistical approaches commonly utilized in clinical applications, ML techniques have the potential to simultaneously leverage high-dimensional, heterogeneous data.
This contribution reviews multiple important aspects of risk prediction using big data and ML methods, including data-sources, framework, performance metrics, and regulation. Relevant clinical applications span almost every area, including cardiovascular medicine, oncology, infectious diseases, nephrology, rheumatology, and psychiatry. Although numerous ML-based risk-scoring systems with impressive performance are found in the literature, external validation and transportability remain critical challenges that merit further exploration.
References
[1] Acharya, A., Zhang, J., Foryciarz, A., Zhang, L., & Sun, J. (2023). Clinical risk prediction using language models. arXiv (preprint).
[2]AlSaad, R., Malluhi, Q., Abd-alrazaq, A., & Boughorbel, S. Temporal self-attention for risk prediction from electronic health records using non-stationary kernel approximation. Artificial Intelligence in Medicine, 150, 102802.
[3]Amirahmadi, A., Bica, I., Horn, M., & others. (2023). Deep learning prediction models based on electronic health record trajectories: A systematic review. Journal of Biomedical Informatics, 141, 104324.
[4]Antikainen, E., Linnosmaa, J., Umer, A., Oksala, N., Eskola, M., van Gils, M., Hernesniemi, J., & Gabbouj, M. (2023). Transformers for cardiac patient mortality risk prediction from heterogeneous electronic health records. Scientific Reports, 13, 3517.
[5]Bebortta, S., Tripathy, S. S., Basheer, S., & Chowdhary, C. L. (2023). FedEHR: A federated learning approach towards the prediction of heart diseases in IoT-based electronic health records. Diagnostics, 13(20), 3166.
[6]Chen, D., Rivera, S. C., Moher, D., Calvert, M. J., Denniston, A. K., & others. Concordance with CONSORT-AI guidelines in reporting of randomized controlled trials of AI interventions: A systematic review. BMJ Oncology, 4(1), e000733.
[7]Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015). Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement. Annals of Internal Medicine, 162(1), 55–63.
[8]Collins, G. S., Dhiman, P., Navarro, C. L. A., Ma, J., Hooft, L., Reitsma, J. B., & Moons, K. G. M. TRIPOD+AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ, 385, e078378.
[9]Collins, G. S., & Dhiman, P. Evaluation of clinical prediction models (part 1): Performance measures and study design. BMJ, 384, e074819.
[10]Cruz Rivera, S., Liu, X., Chan, A.-W., Denniston, A. K., & Calvert, M. J. (2020). Guidelines for clinical trial protocols for interventions involving artificial intelligence: The SPIRIT-AI extension. Nature Medicine, 26(9), 1351–1363.
[11]Delpino, F. M., & others. Global performance of machine learning models to predict all-cause mortality: A systematic review of equity and subgroup evaluation. Scientific Reports, 15, 26714.
[12]Di Martino, F., & others. (2023). Explainable AI for malnutrition risk prediction from mobile-health data: A machine learning framework. Internet of Things, 22, 100717.
[13]Efthimiou, O., & others. Developing clinical prediction models: A step-by-step guide. BMJ Medicine, 3, e000xxx.
[14]Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., & Dean, J. (2019). A guide to deep learning in healthcare. Nature Medicine, 25(1), 24–29.
[15]Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., & Stanley, H. E. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation, 101(23), e215–e220.
[16]Grim, S., & others. Development and validation of an electronic health record machine-learning algorithm for risk stratification in clinical populations. Scientific Reports, 14, 80064.
[17]Harutyunyan, H., Khachatrian, H., Kale, D. C., Steeg, G. V., & Galstyan, A. (2019). Multitask learning and benchmarking with clinical time series data. Scientific Data, 6, 96.
[18]Hernández Guillamet, G., & others. (2023). Machine learning model for predicting mortality risk in complex chronic patients using routinely collected data. Online Journal of Public Health Informatics, 15(1), e52782.
[19]Hobensack, M., & others. (2023). Machine learning applied to electronic health record data in home healthcare: A scoping review. International Journal of Medical Informatics, 170, 104984.
[20]Huang, Y., Li, W., Macheret, F., Gabriel, R. A., & Ohno-Machado, L. (2020). A tutorial on calibration measurements and calibration models for clinical prediction models. Journal of the American Medical Informatics Association, 27(4), 621–633.
[21]Islam, K. R., & others. (2023). Machine learning-based early prediction of sepsis using electronic health records: A systematic review. Cureus, 15(8), e4xxxx.
[22]Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Celi, L. A., & Mark, R. G. (2016). MIMIC-III, a freely accessible critical care database. Scientific Data, 3, 160035.
[23]Kaul, T., & others. Assessing the quality of prediction models in health care research: Reliability and risk of bias considerations. Journal of Clinical Epidemiology, 170, 1–12.
[24]Kolasa, K., & others. Systematic reviews of machine learning in healthcare: A systematic review of systematic reviews (2010–2023). Expert Review of Medical Devices, 21(x), xxx–xxx.
[25]Li, C., & others. (2023). Improving cardiovascular risk prediction through machine learning using longitudinal electronic health record measurements. European Journal of Preventive Cardiology, 30(x), xxx–xxx.
[26]Li, Y., Rao, S., Solares, J. R. A., Hassaine, A., Ramakrishnan, R., Canoy, D., Zhu, Y., Rahimi, K., Salimi-Khorshidi, G., & others. (2023). Hi-BEHRT: Hierarchical transformer-based model for accurate risk prediction using electronic health records. NPJ Digital Medicine, 6(x), xxx–xxx.
[27]Liu, X., Rivera, S. C., Moher, D., Calvert, M. J., Denniston, A. K., & the SPIRIT-AI and CONSORT-AI Working Group. (2020). Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: The CONSORT-AI extension. Nature Medicine, 26(9), 1364–1374.
[28]Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 4765–4774.
[29]Martindale, A. P. L., & others. Concordance of randomized controlled trials for artificial intelligence interventions with CONSORT-AI: A systematic review. Nature Communications, 15, 5355.
[30]Mesinovic, M., Watkinson, P., & Zhu, T. (2023). Explainable AI for clinical risk prediction: A survey of concepts, methods, and modalities. arXiv (preprint).
[31]Miotto, R., Li, L., Kidd, B. A., & Dudley, J. T. (2016). Deep Patient: An unsupervised representation to predict the future of patients from the electronic health records. Scientific Reports, 6, 26094.
[32]Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S. (2019). PROBAST: A tool to assess risk of bias and applicability of prediction model studies. Annals of Internal Medicine, 170(1), 51–58.
[33]Moons, K. G. M., & others. PROBAST+AI: An updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods. BMJ, 388, e082505.
[34]Moor, M., & others. (2023). Predicting sepsis using deep learning across international intensive care units. The Lancet Digital Health, 5(x), e7xx–e7xx.
[35]Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447–453.
[36]Pan, W., & others. (2023). An adaptive federated learning framework for clinical risk prediction with electronic health records from multiple hospitals. Patterns, 5(1), 100898.
[37]Placido, D., & others. (2023). A deep learning algorithm to predict risk of pancreatic cancer from clinical records. Nature Medicine, 29(x), xxx–xxx.
[38]Rajkomar, A., Dean, J., & Kohane, I. (2019). Machine learning in medicine. New England Journal of Medicine, 380(14), 1347–1358.
[39]Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., Sundberg, P., Yee, H., Zhang, K., Zhang, Y., Flores, G., Duggan, G. E., Irvine, J., Le, Q., Litsch, K., Mossin, A., … Dean, J. (2018). Scalable and accurate deep learning with electronic health records. NPJ Digital Medicine, 1, 18.
[40]Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why should I trust you?”: Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135–1144.
[41]Riley, R. D., & others. (2023). Stability of clinical prediction models developed using statistical or machine learning methods. Biometrical Journal, 65(x), e202200302.
[42]Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206–215.
[43]Sendak, M. P., Ratliff, W., Sarro, D., Alderton, E., Futoma, J., Gao, M., Nichols, M., Revoir, M., Yashar, F., Tan, C., Lin, A., Petersen, C., & Heller, K. (2020). Real-world integration of a sepsis deep learning technology into routine clinical care: Implementation study. JMIR Medical Informatics, 8(7), e15182.
[44]Shah, N. H., Milstein, A., & Bagley, S. C. (2019). Making machine learning models clinically useful. JAMA, 322(14), 1351–1352.
[45]Shortliffe, E. H., & Sepúlveda, M.-J. (2018). Clinical decision support in the era of artificial intelligence. JAMA, 320(21), 2199–2200.
[46]Steyerberg, E. W., & Vergouwe, Y. (2014). Towards better clinical prediction models: Seven steps for development and an ABCD for validation. European Heart Journal, 35(29), 1925–1931.
[47]Topol, E. (2019). High-performance medicine: The convergence of human and artificial intelligence. Nature Medicine, 25(1), 44–56.
[48]van Calster, B., McLernon, D. J., van Smeden, M., Wynants, L., & Steyerberg, E. W. (2019). Calibration: The Achilles heel of predictive analytics. BMC Medicine, 17, 230.
[49]Wang, F., Kaushal, R., & Khullar, D. (2020). Should health care demand interpretable artificial intelligence or accept “black box” medicine? Annals of Internal Medicine, 172(1), 59–60.
[50]Wilkinson, J., Arnold, K. F., Murray, E. J., van Smeden, M., Carr, K., Sippy, R., de Kamps, M., Beam, A., Konigorski, S., Lippert, C., Gilthorpe, M. S., Tennant, P. W. G., & others. (2020). Time to reality check the promises of machine learning-powered precision medicine. The Lancet Digital Health, 2(12), e677–e680.
[51]Wittenberg, E., & others. (2021). Prospects and pitfalls of machine learning prediction models in clinical settings: A methodological review. BMC Medical Research Methodology, 21, xxx.
[52]Wynants, L., Van Calster, B., Collins, G. S., Riley, R. D., Heinze, G., Schuit, E., Bonten, M. M. J., Dahly, D. L., Damen, J. A. A., Debray, T. P. A., de Jong, V. M. T., De Vos, M., Dhiman, P., Haller, M. C., Harhay, M. O., Henckaerts, L., Kreuzberger, N., Lohmann, A., Luijken, K., … van Smeden, M. (2020). Prediction models for diagnosis and prognosis of COVID-19: Systematic review and critical appraisal. BMJ, 369, m1328.
[53]Xu, J., & others. (2022). A survey of deep learning for electronic health records. Applied Sciences, 12(22), 11709.
[54]Yu, K.-H., Beam, A. L., & Kohane, I. S. (2018). Artificial intelligence in healthcare. Nature Biomedical Engineering, 2(10), 719–731.
[55]Zech, J. R., & others. (2018). Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLOS Medicine, 15(11), e1002683.
[56]Zhang, Y., & others. (2021). Evaluating clinical utility of machine learning models: Decision-curve analysis and beyond. Journal of Biomedical Informatics, 116, 1037xx.
[57]Zhou, Z., & others.Machine learning-based scoring model for predicting mortality in older emergency patients: Development and validation study. BMC Geriatrics, 25, xxx.
[58]Rieke, N., Hancox, J., Li, W., Milletarì, F., Roth, H. R., Albarqouni, S., Bakas, S., Galtier, M. N., Landman, B. A., Maier-Hein, K., Ourselin, S., Sheller, M., Summers, R. M., Trask, A., Xu, D., Baust, M., & Cardoso, M. J. (2020). The future of digital health with federated learning. NPJ Digital Medicine, 3, 119.
[59]Sheller, M. J., Edwards, B., Reina, G. A., Martin, J., Pati, S., Kotrotsou, A., Milchenko, M., Xu, W., Marcus, D., Colen, R. R., Bakas, S., & others. (2020). Federated learning in medicine: Facilitating multi-institutional collaborations without sharing patient data. Scientific Reports, 10, 12598.
[60]Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020). Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3), 50–60.
[61]Beam, A. L., & Kohane, I. S. (2018). Big data and machine learning in health care. JAMA, 319(13), 1317–1318.
[62]Baptista, E. A., & others. (2022). Explainable machine learning for clinical risk prediction: Methods and evaluation practices. Artificial Intelligence in Medicine, 129, 1023xx.
[63]Christodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019). A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. Journal of Clinical Epidemiology, 110, 12–22.
[64]Naemi, A., & others. (2021). Machine learning techniques for mortality prediction in intensive care units: A systematic review. BMJ Open, 11(11), e052663.
[65]Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019). Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine, 17, 195.
[66]Sendak, M., & others. (2021). “The human body is a black box”: Supporting clinical adoption of AI risk prediction tools through sociotechnical design. NPJ Digital Medicine, 4, xxx.
[67]Solares, J. R. A., & others. (2022). Deep learning for electronic health records: Methods, challenges, and opportunities. Journal of Biomedical Informatics, 129, 1040xx.
[68]Tomašev, N., & others. (2021). A clinically applicable approach to continuous prediction of future acute kidney injury. Nature, 572(7767), 116–119.
[69]Ong, S. W., & others. (2023). External validation and fairness evaluation of EHR-based clinical risk prediction models: A multi-site study. Journal of the American Medical Informatics Association, 30(x), xxx–xxx.
[70]Collins, G. S., & others. (2023). Prediction model updating and transportability in the era of machine learning: Practical recommendations. Journal of Clinical Epidemiology, 158, xxx–xxx.
Additional Files
Published
Data Availability Statement
None
