PROBAST+AI: an introduction to the quality, risk of bias, and applicability assessment tool for prediction model studies using artificial intelligence or regression methods_Chinese Journal of Evidence-Based Medicine

Authors：

LI Zhifan ¹ , LIU Ming ^2,3 , ZHAO Wenshuo ^4,5 , SHI Jiyuan ⁶ , XUE Fuzhong ^4,5 ,  TIAN Jinhui ^2,3 ,  GAO Ya ^4,5

1. Department of Radiology, Shandong Provincial Third Hospital, Cheeloo College of Medicine, Shandong University, Jinan 250031, P. R. China;
2. Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, P. R. China;
3. Key Laboratory of Evidence-Based Medicine of Gansu Province, Lanzhou 730000, P. R. China;
4. Department of Medical Data, School of Public Health, Cheeloo College of Medicine, Shandong University, Jinan 250003, P. R. China;
5. National Institute of Health Data Science of China, Jinan 250003, P. R. China;
6. School of Nursing, Beijing University of Traditional Chinese Medicine, Beijing 100029, P. R. China;

Corresponding author：

TIAN Jinhui, Email: tianjh@lzu.edu.cn; GAO Ya, Email: gaoy2021@163.com

Keywords：

Prediction model; Artificial intelligence; Prognostic model; Diagnostic model; PROBAST; Risk of bias

DOI：

10.7507/1672-2531.202504125

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

With the rapid development of artificial intelligence (AI) and machine learning technologies, the development of AI-based prediction models has become increasingly prevalent in the medical field. However, the PROBAST tool, which is used to evaluate prediction models, has shown growing limitations when assessing models built on AI technologies. Therefore, Moons and colleagues updated and expanded PROBAST to develop the PROBAST+AI tool. This tool is suitable for evaluating prediction model studies based on both artificial intelligence methods and regression methods. It covers four domains: participants and data sources, predictors, outcomes, and analysis, allowing for systematic assessment of quality in model development, risk of bias in model evaluation, and applicability. This article interprets the content and evaluation process of the PROBAST+AI tool, aiming to provide reference and guidance for domestic researchers using this tool.

1.	Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ, 2024, 385: e078378.
2.	Snell KIE, Levis B, Damen JAA, et al. Transparent reporting of multivariable prediction models for individual prognosis or diagnosis: checklist for systematic reviews and meta-analyses (TRIPOD-SRMA). BMJ, 2023, 381: e073538.
3.	Damen JA, Hooft L, Schuit E, et al. Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ, 2016, 353: i2416.
4.	Bellou V, Belbasis L, Konstantinidis AK, et al. Prognostic models for outcome prediction in patients with chronic obstructive pulmonary disease: systematic review and critical appraisal. BMJ, 2019, 367: l5358.
5.	Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ, 2020, 369: m1328.
6.	White N, Parsons R, Collins G, et al. Evidence of questionable research practices in clinical prediction models. BMC Med, 2023, 21(1): 339.
7.	Bouwmeester W, Zuithoff NP, Mallett S, et al. Reporting and methods in clinical prediction research: a systematic review. PLoS Med, 2012, 9(5): 1-12.
8.	Debray TP, Damen JA, Snell KI, et al. A guide to systematic review and meta-analysis of prediction model performance. BMJ, 2017, 356: i6460.
9.	Wolff RF, Moons KGM, Riley RD, et al. PROBAST: A tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med, 2019, 170(1): 51-58.
10.	Moons KGM, Wolff RF, Riley RD, et al. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med, 2019, 170(1): W1-W33.
11.	陈香萍, 张奕, 庄一渝, 等. PROBAST: 诊断或预后多因素预测模型研究偏倚风险的评估工具. 中国循证医学杂志, 2020, 20(6): 737-744.
12.	Moons KGM, Damen JAA, Kaul T, et al. PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods. BMJ, 2025, 388: e082505.
13.	冯诚怿, 张丽伟, 刘惕, 等. 糖尿病合并肺炎克雷伯菌肝脓肿患者发生侵袭综合征预测模型的建立与验证. 中华医学杂志, 2024, 104(12): 956-962.
14.	Šinkovec H, Heinze G, Blagus R, et al. To tune or not to tune, a case study of ridge logistic regression in small or sparse datasets. BMC Med Res Methodol, 2021, 21(1): 199.
15.	Riley RD, Snell KIE, Martin GP, et al. Penalization and shrinkage methods produced unreliable clinical prediction models especially when sample size was small. J Clin Epidemiol, 2021, 132: 88-96.
16.	Van Calster B, van Smeden M, De Cock B, et al. Regression shrinkage methods for clinical prediction models do not guarantee improved performance: simulation study. Stat Methods Med Res, 2020, 29(11): 3166-3178.
17.	Collins GS, Ogundimu EO, Cook JA, et al. Quantifying the impact of different approaches for handling continuous predictors on the performance of a prognostic model. Stat Med, 2016, 35(23): 4124-4135.

1. Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ, 2024, 385: e078378.
2. Snell KIE, Levis B, Damen JAA, et al. Transparent reporting of multivariable prediction models for individual prognosis or diagnosis: checklist for systematic reviews and meta-analyses (TRIPOD-SRMA). BMJ, 2023, 381: e073538.
3. Damen JA, Hooft L, Schuit E, et al. Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ, 2016, 353: i2416.
4. Bellou V, Belbasis L, Konstantinidis AK, et al. Prognostic models for outcome prediction in patients with chronic obstructive pulmonary disease: systematic review and critical appraisal. BMJ, 2019, 367: l5358.
5. Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ, 2020, 369: m1328.
6. White N, Parsons R, Collins G, et al. Evidence of questionable research practices in clinical prediction models. BMC Med, 2023, 21(1): 339.
7. Bouwmeester W, Zuithoff NP, Mallett S, et al. Reporting and methods in clinical prediction research: a systematic review. PLoS Med, 2012, 9(5): 1-12.
8. Debray TP, Damen JA, Snell KI, et al. A guide to systematic review and meta-analysis of prediction model performance. BMJ, 2017, 356: i6460.
9. Wolff RF, Moons KGM, Riley RD, et al. PROBAST: A tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med, 2019, 170(1): 51-58.
10. Moons KGM, Wolff RF, Riley RD, et al. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med, 2019, 170(1): W1-W33.
11. 陈香萍, 张奕, 庄一渝, 等. PROBAST: 诊断或预后多因素预测模型研究偏倚风险的评估工具. 中国循证医学杂志, 2020, 20(6): 737-744.
12. Moons KGM, Damen JAA, Kaul T, et al. PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods. BMJ, 2025, 388: e082505.
13. 冯诚怿, 张丽伟, 刘惕, 等. 糖尿病合并肺炎克雷伯菌肝脓肿患者发生侵袭综合征预测模型的建立与验证. 中华医学杂志, 2024, 104(12): 956-962.
14. Šinkovec H, Heinze G, Blagus R, et al. To tune or not to tune, a case study of ridge logistic regression in small or sparse datasets. BMC Med Res Methodol, 2021, 21(1): 199.
15. Riley RD, Snell KIE, Martin GP, et al. Penalization and shrinkage methods produced unreliable clinical prediction models especially when sample size was small. J Clin Epidemiol, 2021, 132: 88-96.
16. Van Calster B, van Smeden M, De Cock B, et al. Regression shrinkage methods for clinical prediction models do not guarantee improved performance: simulation study. Stat Methods Med Res, 2020, 29(11): 3166-3178.
17. Collins GS, Ogundimu EO, Cook JA, et al. Quantifying the impact of different approaches for handling continuous predictors on the performance of a prognostic model. Stat Med, 2016, 35(23): 4124-4135.

Chinese Journal of Evidence-Based Medicine

Latest ArticlesPROBAST+AI: an introduction to the quality, risk of bias, and applicability assessment tool for prediction model studies using artificial intelligence or regression methods

Abstract Full text Figures/Tables Video References Cited by

Format

Content