Advertisement

Influence of threshold selection strategy on the prognostic accuracy of chest CT severity score for mortality prediction of COVID-19 patients

      To the Editor,
      Coronavirus disease 2019 (COVID-19) is a new wave of emerging infections that quickly spread and its pandemic was declared as an outbreak of a global health emergency of international concern on January 30, 2020.
      • Ng M-Y
      • Lee EY
      • Yang J
      • et al.
      Imaging profile of the COVID-19 infection: radiologic findings and literature review.
      ,
      • Li Y
      • Xia L
      Coronavirus disease 2019 (COVID-19): role of chest CT in diagnosis and management.
      As of June 23, 2022, 546725797 laboratory-confirmed cases have been globally documented

      Worldometers. United States [June 23, 2022, 13:25 GMT]. Available from: https://www.worldometers.info/coronavirus/.

      and the sharp increase in the case numbers makes this more complicated. In COVID-19 patients, the clinical spectrum of COVID-19 can be seen ranging from asymptomatic to severe or critical. Approximately 80 percent are asymptomatic or with mild symptoms, 15% have severe disease, and 5% become critical.
      • Rokni M
      • Ghasemi V
      • Tavakoli Z.
      Immune responses and pathogenesis of SARS-CoV-2 during an outbreak in Iran: comparison with SARS and MERS.
      During the major outbreak of the disease, the role of chest computed tomography (CT) as a diagnostic tool has been already verified.
      • Pan Y
      • Guan H
      • Zhou S
      • et al.
      Initial CT findings and temporal changes in patients with the novel coronavirus pneumonia (2019-nCoV): a study of 63 patients in Wuhan, China.
      • Fang Y
      • Zhang H
      • Xie J
      • et al.
      Sensitivity of chest CT for COVID-19: comparison to RT-PCR.
      • Xie X
      • Zhong Z
      • Zhao W
      • Zheng C
      • Wang F
      • Liu J.
      Chest CT for typical 2019-nCoV pneumonia: relationship to negative RT-PCR testing.
      The chest CT as a first-line diagnostic tool could play a critical role in the detection, evaluation of pulmonary extension, evaluation of disease severity, and monitoring of the disease activity. CT severity score (CT-SS) is determined according to the extent of lung involvement on the CT images and is an appropriate prognostic factor for mortality prediction in patients with COVID-19 pneumonia.
      • Zakariaee SS
      • Naderi N
      • Rezaee D.
      Prognostic accuracy of visual lung damage computed tomography score for mortality prediction in patients with COVID-19 pneumonia: a systematic review and meta-analysis.
      In Cao Y et al. study,
      • Cao Y
      • Han X
      • Gu J
      • et al.
      Prognostic value of baseline clinical and HRCT findings in 101 patients with severe COVID-19 in Wuhan.
      it was reported that deceased patients had higher CT-SSs than discharged patients (20.9 ± 3.0 vs. 15.6 ± 5.0, p < 0.001). Similar results were also observed in several countries.
      • Bayrak V
      • Durukan NŞ
      • Aydemir FD
      • et al.
      Risk factors associated with mortality in ıntensive care COVID-19 patients: the importance of chest CT score and intubation timing as risk factors.
      • Charpentier E
      • Soulat G
      • Fayol A
      • et al.
      Visual lung damage CT score at hospital admission of COVID-19 patients and 30-day mortality.
      • Chon Y
      • Kim JY
      • Suh YJ
      • et al.
      Adverse initial CT findings associated with poor prognosis of coronavirus disease.
      • Francone M
      • Iafrate F
      • Masci GM
      • et al.
      Chest CT score in COVID-19 patients: correlation with disease severity and short-term prognosis.
      • Hajiahmadi S
      • Shayganfar A
      • Janghorbani M
      • et al.
      Chest computed tomography severity score to predict adverse outcomes of patients with COVID-19.
      These studies suggest that the patients with higher CT-SSs might have more severe clinical outcomes and are more susceptible to mortality. Determination of the appropriate classification cut-off for this prognostic factor could have a considerable role in the early diagnosis and management of patients with poor prognoses. In Khosravi et al. study,
      • Khosravi B
      • Aghaghazvini L
      • Sorouri M
      • et al.
      Predictive value of initial CT scan for various adverse outcomes in patients with COVID-19 pneumonia.
      the median of the CT-SSs was used as the discriminative threshold for all analyses. We are skeptical regarding the sensitivity and performance of the selected threshold for mortality prediction of COVID-19 patients. For a classification test, the receiver operating characteristic (ROC) curve is the most commonly used method to determine the best cut-off value.
      • Habibzadeh F
      • Habibzadeh P
      • Yadollahie M.
      On determining the most appropriate test cut-off value: the case of tests with continuous results.
      In this note, we aim to compare the prognostic performance of CT-SSs based on median and ROC-based selected thresholds. Hence, we re-analyzed the prognostic performance of CT-SSs reported by Cao Y et al.
      • Cao Y
      • Han X
      • Gu J
      • et al.
      Prognostic value of baseline clinical and HRCT findings in 101 patients with severe COVID-19 in Wuhan.
      and Li K et al.
      • Li K
      • Chen D
      • Chen S
      • et al.
      Predictors of fatality including radiographic findings in adults with COVID-19.
      In these evaluations, the prognostic accuracies of CT-SSs were determined for the median and ROC-based selected thresholds. The prognostic performances of median and ROC-based selected thresholds of the CT scores are listed in Table 1. The ROC curves are depicted in Figure 1. For CT-SSs reported by Li K et al., the ROC-based selected threshold improved all parameters of the prognosis performance. By taking the ROC-based selected threshold for CT-SSs reported by Cao Y et al., a higher number of deceased patients could be detected. As it could be concluded from the results, ROC-based selected thresholds have higher sensitivities and better performances to discriminate the patients with poor prognosis. The threshold selection strategy has a considerable influence on the prognostic accuracy of CT-SS for mortality prediction of COVID-19 patients. Given the substantial impact of COVID-19 on global health and the importance of risk stratification for the allocation of finite resources such as antivirals and intensive care beds, it is recommended that the ROC-based strategy be used to select the optimal CT-SS threshold for screening patients with poor prognosis in triage.
      Table 1Prognostic performances of median and ROC-based selected thresholds for mortality prediction of COVID-19 patients.
      StudyThreshold TypeCut-offAccuracySensitivitySpecificityPPredNPredAUC
      Cao Y et al.Median1972.2868.5774.2458.5481.678110.4
      ROC-based selected threshold18-18.7568.3277.1463.6452.9484.00
      Li K et al.Median8.578.1290.9171.4362.5093.758549.8
      ROC-based selected threshold6.09-7.9881.25100.0071.4364.71100.00
      PPred: Positive Predictive value; NPred: Negative Predictive value; AUC: The area under the curve.
      Fig 1
      Fig. 1a) ROC curve plotted based on the CT-SSs reported by Li K et al. b) ROC curve plotted based on the CT-SSs reported by Cao Y et al.

      Declaration of Competing Interest

      The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

      Acknowledgment

      Authors would like to thank the research affair of Ilam University of Medical Sciences.

      References

        • Ng M-Y
        • Lee EY
        • Yang J
        • et al.
        Imaging profile of the COVID-19 infection: radiologic findings and literature review.
        Radiology. 2020; 2: 1-9
        • Li Y
        • Xia L
        Coronavirus disease 2019 (COVID-19): role of chest CT in diagnosis and management.
        AJR Am J Roentgenol. 2020; 214: 1280-1286
      1. Worldometers. United States [June 23, 2022, 13:25 GMT]. Available from: https://www.worldometers.info/coronavirus/.

        • Rokni M
        • Ghasemi V
        • Tavakoli Z.
        Immune responses and pathogenesis of SARS-CoV-2 during an outbreak in Iran: comparison with SARS and MERS.
        Rev Med Virol. 2020; 30: 1-6
        • Pan Y
        • Guan H
        • Zhou S
        • et al.
        Initial CT findings and temporal changes in patients with the novel coronavirus pneumonia (2019-nCoV): a study of 63 patients in Wuhan, China.
        Eur radiol. 2020; 30: 3306-3309
        • Fang Y
        • Zhang H
        • Xie J
        • et al.
        Sensitivity of chest CT for COVID-19: comparison to RT-PCR.
        Radiology. 2020; 296: 115-117
        • Xie X
        • Zhong Z
        • Zhao W
        • Zheng C
        • Wang F
        • Liu J.
        Chest CT for typical 2019-nCoV pneumonia: relationship to negative RT-PCR testing.
        Radiology. 2020; 296: 41-45
        • Zakariaee SS
        • Naderi N
        • Rezaee D.
        Prognostic accuracy of visual lung damage computed tomography score for mortality prediction in patients with COVID-19 pneumonia: a systematic review and meta-analysis.
        Egypt J Radiol Nucl Med. 2022; 53: 1-9
        • Cao Y
        • Han X
        • Gu J
        • et al.
        Prognostic value of baseline clinical and HRCT findings in 101 patients with severe COVID-19 in Wuhan.
        China. Sci Rep. 2020; 10: 1-3
        • Bayrak V
        • Durukan NŞ
        • Aydemir FD
        • et al.
        Risk factors associated with mortality in ıntensive care COVID-19 patients: the importance of chest CT score and intubation timing as risk factors.
        Turk J Med Sci. 2021; 51: 1665-1674
        • Charpentier E
        • Soulat G
        • Fayol A
        • et al.
        Visual lung damage CT score at hospital admission of COVID-19 patients and 30-day mortality.
        Eur Radiol. 2021; 31: 8354-8363
        • Chon Y
        • Kim JY
        • Suh YJ
        • et al.
        Adverse initial CT findings associated with poor prognosis of coronavirus disease.
        J Korean Med Sci. 2020; 35: 1-12
        • Francone M
        • Iafrate F
        • Masci GM
        • et al.
        Chest CT score in COVID-19 patients: correlation with disease severity and short-term prognosis.
        Eur Radiol. 2020; 30: 6808-6817
        • Hajiahmadi S
        • Shayganfar A
        • Janghorbani M
        • et al.
        Chest computed tomography severity score to predict adverse outcomes of patients with COVID-19.
        Infect Chemother. 2021; 53: 308-318
        • Khosravi B
        • Aghaghazvini L
        • Sorouri M
        • et al.
        Predictive value of initial CT scan for various adverse outcomes in patients with COVID-19 pneumonia.
        Heart Lung. 2021; 50: 13-20
        • Habibzadeh F
        • Habibzadeh P
        • Yadollahie M.
        On determining the most appropriate test cut-off value: the case of tests with continuous results.
        Biochem Med. 2016; 26: 297-307
        • Li K
        • Chen D
        • Chen S
        • et al.
        Predictors of fatality including radiographic findings in adults with COVID-19.
        Respir Res. 2020; 21: 1-10