Development and external validation of models for predicting mortality, sepsis and septic shock in COVID-19 patients

Noskov A.A.; Zabolotskikh I.B.

doi:https://doi.org/10.17116/anaesthesiology202603125

OBJECTIVE

To develop and externally validate the models for predicting in-hospital mortality and septic complications in patients with COVID-19.

MATERIAL AND METHODS

This study was conducted in accordance with the TRIPOD guidelines. A retrospective derivation cohort included patients hospitalized between April 2020 and December 2021. External validation was performed in a cohort of patients hospitalized during the period of Omicron variant predominance (January—February 2022).

RESULTS

The final model included age, SpO₂, respiratory rate, and Glasgow Coma Scale score. The derivation cohort included 488 patients, while the validation cohort comprised 382 patients. During external validation, AUROC for in-hospital mortality was 0.798 (95% CI 0.752—0.845), while AUROC for sepsis and septic shock were 0.680 and 0.704, respectively. Calibration was good for mortality and less pronounced for other outcomes.

CONCLUSION

Original model demonstrates stable discriminative performance and acceptable calibration for predicting in-hospital mortality in patients with COVID-19.

Keywords:

COVID-19

prognostic model

adverse outcome

validation

Authors:

Noskov A.A.

Kuban State Medical University, Krasnodar, Russia;
Krasnodar Regional Clinical Hospital No. 2, Krasnodar, Russia

ORCID: 0000-0002-0662-9358

Zabolotskikh I.B.

Kuban State Medical University, Krasnodar, Russia;
Krasnodar Regional Clinical Hospital No. 2, Krasnodar, Russia;
Federal Research Clinical Center of Intensive Care and Rehabilitology, Moscow, Russia

SPIN RSCI: 5142-1661
Scopus AuthorID: 6508104506
ResearcherID: O-2705-2014
ORCID: 0000-0002-3623-2546

Received:

27.01.2026

Accepted:

12.03.2026

Close metadata

References:

Soriano JB, Murthy S, Marshall JC, Relan P, Diaz JV; WHO Clinical Case Definition Working Group on Post-COVID-19 Condition. A clinical case definition of post-COVID-19 condition by a Delphi consensus. The Lancet. Infectious Diseases. 2022;22(4):e102-e10. https://doi.org/10.1016/S1473-3099(21)00703-9
Wolff D, Nee S, Hickey NS, Marschollek M. Risk factors for Covid-19 severity and fatality: a structured literature review. Infection. 2021;49(1):15-28. https://doi.org/10.1007/s15010-020-01509-1
Nyberg T, Ferguson NM, Nash SG, Webster HH, Flaxman S, Andrews N, Hinsley W, Bernal JL, Kall M, Bhatt S, Blomquist P, Zaidi A, Volz E, Aziz NA, Harman K, Funk S, Abbott S; COVID-19 Genomics UK (COG-UK) consortium; Hope R, Charlett A, Chand M, Ghani AC, Seaman SR, Dabrera G, De Angelis D, Presanis AM, Thelwall S. Comparative analysis of the risks of hospitalisation and death associated with SARS-CoV-2 omicron (B.1.1.529) and delta (B.1.617.2) variants in England: a cohort study. Lancet. 2022;399(10332):1303-1312. https://doi.org/10.1016/S0140-6736(22)00462-7
Iuliano AD, Brunkard JM, Boehmer TK, Peterson E, Adjei S, Binder AM, Cobb S, Graff P, Hidalgo P, Panaggio MJ, Rainey JJ, Rao P, Soetebier K, Wacaster S, Ai C, Gupta V, Molinari NM, Ritchey MD. Trends in Disease Severity and Health Care Utilization During the Early Omicron Variant Period Compared with Previous SARS-CoV-2 High Transmission Periods — United States, December 2020-January 2022. MMWR. Morbidity and Mortality Weekly Report. 2022;71(4):146-152. https://doi.org/10.15585/mmwr.mm7104e4
Sobczak M, Pawliczak R. COVID-19 vaccination efficacy in numbers including SARS-CoV-2 variants and age comparison: a meta-analysis of randomized clinical trials. Annals of Clinical Microbiology and Antimicrobials. 2022;21(1):32. https://doi.org/10.1186/s12941-022-00525-3
Kostakis I, Smith GB, Prytherch D, Meredith P, Price C, Chauhan A; Portsmouth Academic ConsortIum For Investigating COVID-19 (PACIFIC-19). The performance of the National Early Warning Score and National Early Warning Score 2 in hospitalised patients infected by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Resuscitation. 2021;159:150-157. https://doi.org/10.1016/j.resuscitation.2020.10.039
Wang DX, Wang MK, Rochwerg B, Burns KEA. Predictive performance of SOFA & qSOFA for in-hospital mortality in patients with severe novel coronavirus disease. American Journal of Emergency Medicine. 2021;50:793-794. https://doi.org/10.1016/j.ajem.2021.03.009
Bourn SS, Crowe RP, Fernandez AR, Matt SE, Brown AL, Hawthorn AB, Myers JB. Initial prehospital Rapid Emergency Medicine Score (REMS) to predict outcomes for COVID-19 patients. Journal of the American College of Emergency Physicians Open. 2021;2(4):e12483. https://doi.org/10.1002/emp2.12483
Vedovati MC, Barbieri G, Urbini C, D’Agostini E, Vanni S, Papalini C, Pucci G, Cimini LA, Valentino A, Ghiadoni L, Becattini C. Clinical prediction models in hospitalized patients with COVID-19: A multicenter cohort study. Respiratory Medicine. 2022;202:106954. https://doi.org/10.1016/j.rmed.2022.106954
Knight SR, Ho A, Pius R, Buchan I, Carson G, Drake TM, Dunning J, Fairfield CJ, Gamble C, Green CA, Gupta R, Halpin S, Hardwick HE, Holden KA, Horby PW, Jackson C, Mclean KA, Merson L, Nguyen-Van-Tam JS, Norman L, Noursadeghi M, Olliaro PL, Pritchard MG, Russell CD, Shaw CA, Sheikh A, Solomon T, Sudlow C, Swann OV, Turtle LC, Openshaw PJ, Baillie JK, Semple MG, Docherty AB, Harrison EM; ISARIC4C investigators. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score. BMJ. 2020;370:m3339. https://doi.org/10.1136/bmj.m3339
Liang W, Liang H, Ou L, Chen B, Chen A, Li C, Li Y, Guan W, Sang L, Lu J, Xu Y, Chen G, Guo H, Guo J, Chen Z, Zhao Y, Li S, Zhang N, Zhong N, He J; China Medical Treatment Expert Group for COVID-19. Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19. JAMA Internal Medicine. 2020;180(8):1081-1089. https://doi.org/10.1001/jamainternmed.2020.2033
Gupta RK, Harrison EM, Ho A, Docherty AB, Knight SR, van Smeden M, Abubakar I, Lipman M, Quartagno M, Pius R, Buchan I, Carson G, Drake TM, Dunning J, Fairfield CJ, Gamble C, Green CA, Halpin S, Hardwick HE, Holden KA, Horby PW, Jackson C, Mclean KA, Merson L, Nguyen-Van-Tam JS, Norman L, Olliaro PL, Pritchard MG, Russell CD, Scott-Brown J, Shaw CA, Sheikh A, Solomon T, Sudlow C, Swann OV, Turtle L, Openshaw PJM, Baillie JK, Semple MG, Noursadeghi M; ISARIC4C Investigators. Development and validation of the ISARIC 4C Deterioration model for adults hospitalised with COVID-19: a prospective cohort study. Lancet Respiratory Medicine. 2021;9(4):349-359. https://doi.org/10.1016/S2213-2600(20)30559-2
Andrea Gallego Aristizabal P, Paola Lujan Chavarría T, Isabel Vergara Hernández S, Rincón Acosta F, Paula Sánchez Carmona M, Andrea Salazar Ospina P, Jose Atencia Florez C, Mario Barros Liñán C, Jaimes F. External validation of two clinical prediction models for mortality in COVID-19 patients (4C and NEWS2), in three centers in Medellín, Colombia: Assessing the impact of vaccination over time. Infectious Diseases Now. 2024;54(5):104921. https://doi.org/10.1016/j.idnow.2024.104921
Häger L, Wendland P, Biergans S, Lederer S, de Arruda Botelho Herr M, Erhardt C, Schmauder K, Kschischo M, Malek NP, Bunk S, Bitzer M, Gladstone BP, Göpel S. External Validation of COVID-19 Risk Scores during Three Waves of Pandemic in a German Cohort-A Retrospective Study. Journal of Personalized Medicine. 2022;12(11):1775. https://doi.org/10.3390/jpm12111775
Tohyama T, McCoy LG, Ishii E, Sood S, Raffa J, Kinoshita T, Celi LA, Hashimoto S. Mortality Prediction Performance Under Geographical, Temporal, and COVID-19 Pandemic Dataset Shift: External Validation of the Global Open-Source Severity of Illness Score Model. Critical Care Explorations. 2025;7(6):e1275. https://doi.org/10.1097/CCE.0000000000001275
Noskov AA, Glushko EI, Zabolotskikh IB. Rapid Scoring Systems for Predicting Adverse Outcomes in Patients with COVID-19: A Systematic Review and Meta-Analysis. Annals of Critical Care. 2025;(3):141-161. (In Russ). https://doi.org/10.21320/1818-474X-2025-3-141-161
Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Annals of Internal Medicine. 2015;162(1):55-63. https://doi.org/10.7326/M14-0697
Zabolotskikh IB, Kirov MYu, Lebedinskii KM, Protsenko DN, Avdeev SN, Andreenko AA, Arsent’ev LV, Afonchikov VS, Afukov II, Belkin AA, Boeva EA, Bulanov AYu, Vasil’ev Ya., Vlasenko AV, Gorbachev VI, Grigor’ev EV, Grigor’ev SV, Gritsan AI, Eremenko AA, Ershov EN, Zamyatin MN, Kuzovlev AN, Kulikov AV, Lakhin RE, Leiderman IN, Len’kin AI, Mazurok VA, Musaeva TS, Nikolaenko EM, Orlov YuP, Petrikov SS, Roitman EV, Ronenson AM, Smetkin AA, Sokolov AA, Stepanenko SM, Subbotin VV, Ushakova ND, Khoronenko VE, Tsarenko SV, Shifman EM, Shukevich DL, Shchegolev AV, Yaroshetskii AI, Yarustovskii MB. Anesthesia and intensive care for patients with COVID-19. Russian Federation of anesthesiologists and reanimatologists guidelines. Annals of Critical Care. 2020;S1:9-120. (In Russ.). https://doi.org/10.21320/1818-474X-2020-S1-9-120
Zabolotskikh IB, Kirov MYu, Lebedinskii KM, Protsenko DN, Avdeev SN, Andreenko AA, Arsent’ev LV, Afonchikov VS, Afukov II, Belkin AA, Boeva EA, Bulanov AYu, Vasil’ev Ya., Vlasenko AV, Gorbachev VI, Grigor’ev EV, Grigor’ev SV, Gritsan AI, Eremenko AA, Ershov EN, Zamyatin MN, Kuzovlev AN, Kulikov AV, Lakhin RE, Leiderman IN, Len’kin AI, Mazurok VA, Musaeva TS, Nikolaenko EM, Orlov YuP, Petrikov SS, Roitman EV, Ronenson AM, Smetkin AA, Sokolov AA, Stepanenko SM, Subbotin VV, Ushakova ND, Khoronenko VE, Tsarenko SV, Shifman EM, Shukevich DL, Shchegolev AV, Yaroshetskii AI, Yarustovskii MB. Anesthesia and intensive care for patients with COVID-19. Russian Federation of anesthesiologists and reanimatologists guidelines. Annals of Critical Care. 2021;(1-S): 9-143. (In Russ.). https://doi.org/10.21320/1818-474X-2021-S1-9-143
Bihorac A, Ozrazgat-Baslanti T, Ebadi A, Motaei A, Madkour M, Pardalos PM, Lipori G, Hogan WR, Efron PA, Moore F, Moldawer LL, Wang DZ, Hobson CE, Rashidi P, Li X, Momcilovic P. MySurgeryRisk: Development and Validation of a Machine-learning Risk Algorithm for Major Complications and Death After Surgery. Annals of Surgery. 2019;269(4):652-662. https://doi.org/10.1097/SLA.0000000000002706
Choi B, Oh AR, Lee SH, Lee DY, Lee JH, Yang K, Kim HY, Park RW, Park J. Prediction Model for 30-Day Mortality after Non-Cardiac Surgery Using Machine-Learning Techniques Based on Preoperative Evaluation of Electronic Medical Records. Journal of Clinical Medicine. 2022;11(21):6487. https://doi.org/10.3390/jcm11216487
Wijeysundera DN. Predicting outcomes: Is there utility in risk scores?. Canadian Journal of Anesthesia. 2016;63(2):148-158. https://doi.org/10.1007/s12630-015-0537-2
Xie J, Hungerford D, Chen H, Abrams TS. Development and external validation of a prognostic multivariable model for severe COVID-19. BMJ. 2020;371:m3493. https://doi.org/10.1101/2020.03.28.20045997
Wynants L, Van Calster B, Collins GS, Riley RD, Heinze G, Schuit E, Bonten MMJ, Dahly DL, Damen JAA, Debray TPA, de Jong VMT, De Vos M, Dhiman P, Haller MC, Harhay MO, Henckaerts L, Heus P, Kammer M, Kreuzberger N, Lohmann A, Luijken K, Ma J, Martin GP, McLernon DJ, Andaur Navarro CL, Reitsma JB, Sergeant JC, Shi C, Skoetz N, Smits LJM, Snell KIE, Sperrin M, Spijker R, Steyerberg EW, Takada T, Tzoulaki I, van Kuijk SMJ, van Bussel B, van der Horst ICC, van Royen FS, Verbakel JY, Wallisch C, Wilkinson J, Wolff R, Hooft L, Moons KGM, van Smeden M. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ. 2020;369:m1328. https://doi.org/10.1136/bmj.m1328
Sperrin M, Grant SW, Peek N. Prediction models for diagnosis and prognosis in Covid-19. BMJ. 2020;369:m1464. https://doi.org/10.1136/bmj.m1464
Gupta RK, Marks M, Samuels THA, Luintel A, Rampling T, Chowdhury H, Quartagno M, Nair A, Lipman M, Abubakar I, van Smeden M, Wong WK, Williams B, Noursadeghi M; UCLH COVID-19 Reporting Group. Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: an observational cohort study. European Respiratory Journal. 2020;56(6):2003498. https://doi.org/10.1183/13993003.03498-2020
Lombardi Y, Azoyan L, Szychowiak P, Bellamine A, Lemaitre G, Bernaux M, Daniel C, Leblanc J, Riller Q, Steichen O; AP-HP/Universities/INSERM COVID-19 Research Collaboration AP-HP COVID CDR Initiative. External validation of prognostic scores for COVID-19: a multicenter cohort study of patients hospitalized in Greater Paris University Hospitals. Intensive Care Medicine. 2021;47(12):1426-1439. https://doi.org/10.1007/s00134-021-06524-w
Zabolotskikh IB, Zybin KD, Noskov AA, Kuznetsova MP, Musaeva TS, Sinkov SV. Comparison of REMS, NEWS and qSOFA scales in mortality prediction for patients with confirmed SARS-CoV-2 infection upon admission to ICU: a retrospective observational trial. Russian Journal of Anesthesiology and Reanimatology. 2020;(6-2):20-26. (In Russ.). https://doi.org/10.17116/anaesthesiology202006220