Advertisement

Development and validation of a prognostic nomogram model for ICU patients with alcohol-associated cirrhosis

  • Luyan Zheng
    Affiliations
    State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, National Clinical Research Center for Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, 310003, China
    Search for articles by this author
  • Yining Lu
    Affiliations
    State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, National Clinical Research Center for Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, 310003, China
    Search for articles by this author
  • Author Footnotes
    1 Contribute equally.
    Jie Wu
    Correspondence
    Corresponding authors at: #79 Qingchun Road, Hangzhou, Zhejiang Province, P.R. China, 310003.
    Footnotes
    1 Contribute equally.
    Affiliations
    State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, National Clinical Research Center for Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, 310003, China
    Search for articles by this author
  • Author Footnotes
    1 Contribute equally.
    Min Zheng
    Correspondence
    Corresponding authors at: #79 Qingchun Road, Hangzhou, Zhejiang Province, P.R. China, 310003.
    Footnotes
    1 Contribute equally.
    Affiliations
    State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, National Clinical Research Center for Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, 310003, China
    Search for articles by this author
  • Author Footnotes
    1 Contribute equally.
Open AccessPublished:January 22, 2023DOI:https://doi.org/10.1016/j.dld.2023.01.148

      Abstract

      Background

      The prognosis of patients with alcohol-associated cirrhosis (ALC) admitted to the intensive care unit (ICU) is poor. We developed and validated a nomogram (NIALC) for ICU patients with ALC.

      Methods

      Predictors of mortality were defined by a machine learning method in a cohort of 394 ICU patients with ALC from the Medical Information Mart for Intensive Care database. Then the nomogram (NIALC) was constructed and evaluated using the AUC. The MELD, MELD-sodium, Child–Pugh, and CLIF-SOFA scores were then compared with NIALC. Two datasets of 394 and 501 ICU patients with ALC were utilized for model validation.

      Results

      In-hospital mortality was 41% and 21% in the training and external validation sets. Predictors included were blood urea nitrogen, total bilirubin, prothrombin time, serum creatinine, lactate, partial thromboplastin time, phosphate, mean arterial pressure, lymphocytes, fibrinogen, and albumin. The AUCs for the NIALC were 0.767 and 0.760 in the two validation cohorts, which were better than those of the MELD, MELD-sodium, Child–Pugh, and CLIF-SOFA.

      Conclusion

      We developed a nomogram for ICU patients with ALC, which demonstrated better discriminative ability than previous prognostic scores. This nomogram could be conveniently used to facilitate the individualized prediction of death in ICU patients with ALC.

      Keywords

      1. Introduction

      The Global Burden of Disease study estimated that there were 1.47 million deaths in 2019 due to cirrhosis and chronic liver disease worldwide, and alcohol-related cirrhosis (ALC) represented 47.9% of all liver cirrhosis deaths [
      • Rehm J.
      • Samokhvalov A.V.
      • Shield K.D
      Global burden of alcoholic liver diseases.
      ]. Alcohol is a predominant cause of liver cirrhosis in Europe, the United States and other Western countries, and the contribution of alcohol to cirrhosis is also increasing in China and other Asian countries [
      • Seitz H.K.
      • Bataller R.
      • Cortez-Pinto H.
      • Gao B.
      • Gual A.
      • Lackner C.
      • et al.
      Alcoholic liver disease.
      ,
      • Kevin D
      • Shield M.R.
      • R J.
      Public health successes and missed opportunities.
      . Liver cirrhosis evolves from an asymptomatic phase (compensated cirrhosis) to a symptomatic phase (decompensated cirrhosis, [DC]). Patients with DC tend to be admitted to intensive care units (ICUs) for severe complications such as gastrointestinal bleeding (GIB), acute kidney injury (AKI), and hepatic encephalopathy (HE). The overall prognosis for patients with DC admitted to the ICU is poor, with mortality rates ranging from 44 to 74% [
      • Cholongitas E.
      • Senzolo M.
      • Patch D.
      • Shaw S.
      • Hui C.
      • Burroughs A.K
      Review article: scoring systems for assessing prognosis in critically ill adult cirrhotics.
      ].
      In ICU patients with DC, multiple organ failure caused by severe complications is the most common cause of death. Since there are many therapies that could prevent the development of organ failure, it is vital to develop an accurate prognostic score for these patients and identify patients who are most likely to benefit from targeted preventive therapy [
      • Ginès P.
      • Fernández J.
      • Durand F.
      • Saliba F
      Management of critically-ill cirrhotic patients.
      ]. Although several prognostic scores have been proposed from cohorts of ICU patients with cirrhosis [
      • Cholongitas E.
      • Senzolo M.
      • Patch D.
      • Kwong K.
      • Nikolopoulou V.
      • Leandro G.
      • et al.
      Risk factors, sequential organ failure assessment and model for end-stage liver disease scores for predicting short term mortality in cirrhotic patients admitted to intensive care unit.
      ,
      • Fang J.-.T.
      • Tsai M.-.H.
      • Tian Y.-.C.
      • Jenq C.-.C.
      • Lin C.-.Y.
      • Chen Y.-.C.
      • et al.
      Outcome predictors and new score of critically ill cirrhotic patients with acute renal failure.
      ,
      • Zauner C.
      • Schneeweiss B.
      • Schneider B.
      • Madl C.
      • Klos H.
      • Kranz A.
      • et al.
      Short-term prognosis in critically ill patients with liver cirrhosis: an evaluation of a new scoring system.
      ,
      • Theocharidou E.
      • Pieri G.
      • Mohammad A.O.
      • Cheung M.
      • Cholongitas E.
      • Agarwal B.
      • et al.
      The Royal Free Hospital score: a calibrated prognostic model for patients with cirrhosis admitted to intensive care unit. Comparison with current models and CLIF-SOFA score.
      ], these models still have some limitations, for example, the relatively small number of samples, their complicated composition, and inefficient external validation, which together hamper their utilization in routine clinical practice. There is an urgent need to explore more optimized models.
      In addition, many models have been widely utilized for the prognostication of severe liver diseases and they show good performance in clinical practice, including the Child–Pugh, the Model for End-Stage Liver Disease (MELD), the MELD-sodium [
      • Biggins S.W.
      • Kim W.R.
      • Terrault N.A.
      • Saab S.
      • Balan V.
      • Schiano T.
      • et al.
      Evidence-based incorporation of serum sodium concentration into MELD.
      ], and the Chronic Liver Failure-SOFA (CLIF-SOFA) scores. However, they were generally not specifically developed for ICU patients with ALC [
      • Kim H.Y.
      • Kim C.W.
      • Kim T.Y.
      • Song D.S.
      • Sinn D.H.
      • Yoon E.L.
      • et al.
      Assessment of scoring systems for acute-on-chronic liver failure at predicting short-term mortality in patients with alcoholic hepatitis.
      ]. Given substantial heterogeneities in the clinical characteristics between ICU and non-ICU patients, further assessment of these prognostic scores in ICU patients with ALC is warranted. We aimed to establish and validate a novel prognostic model using a nomogram-based approach for predicting in-hospital mortality in ICU patients with ALC. In addition, we compared the performance of the nomogram to the established prognostic models, including the MELD, MELD-sodium, Child–Pugh, and CLIF-SOFA scores.

      2. Materials and methods

      2.1 Data source and study population

      The study population was derived from the Medical Information Mart for Intensive Care (MIMIC) and the eICU Collaborative Research Database (eICU-CRD) databases. Both are public databases with data from ICU patients, which include detailed records of demographics, vital signs, laboratory results, and nursing progress notes from 38,605 and 139,367 ICU patients, respectively [
      • Johnson A.E.W.
      • Pollard T.J.
      • Shen L.
      • Lehman L-WH
      • Feng M.
      • Ghassemi M.
      • et al.
      MIMIC-III, a freely accessible critical care database.
      ,
      • Pollard T.J.
      • Johnson A.E.W.
      • Raffa J.D.
      • Celi L.A.
      • Mark R.G.
      • Badawi O
      The eICU Collaborative Research Database, a freely available multi-center database for critical care research.
      . Additional details about the databases can be found in the Supplementary Methods section. Adult (≥18 years old) patients with a diagnosis of “alcoholic liver cirrhosis” (571.2 according to ICD-9 or K70.3 according to ICD-10) were initially identified from the MIMIC-III and eICU-CRD databases. Pregnant women or those who had recently given birth were excluded from this study. Patients with a concurrent malignant neoplasm were also excluded. Then, the patients from the MIMIC database were randomly split into a training cohort and an internal validation cohort at a 1:1 ratio, while the patients from the eICU-CRD database were used as an external validation cohort. The inclusion and exclusion criteria are summarized in Fig. 1. This study was performed according to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines [
      • Moons K.G.M.
      • Altman D.G.
      • Reitsma J.B.
      • Ioannidis J.P.A.
      • Macaskill P.
      • Steyerberg E.W.
      • et al.
      Transparent reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration.
      ]. No IRB review was necessary (and thus no number was assigned) because it did not fall under the board's guidelines as human subjects research.
      Fig. 1
      Fig. 1Flowchart of patient enrollment. Abbreviations: MIMIC, Medical Information Mark for Intensive Care; ICU, intensive care unit; eICU-CRD, eICU Collaborative Research Database; AKI, acute kidney injury; GIB, gastrointestinal bleeding; SBP, systolic blood pressure; HE, hepatic encephalopathy.

      2.2 Candidate prognostic factors and outcome measures

      The primary outcome for this study was the in-hospital all-cause mortality of patients admitted to the ICU unit. Baseline data of the included patients were collected within 24 h of ICU admission, including demographic data, vital sign data, cirrhosis complications, and laboratory tests. For some variables with multiple measurements within 24 h, the mean or median was used for model development. We initially identified a total of 73 variables for the included patients. Variables with missing values greater than 90% were excluded, and those variables with missing values less than 10% were imputed with the mean or median. We next selected variables using a machine learning regression analysis method, the Least Absolute Shrinkage and Selection Operator (LASSO), to improve the model accuracy and reduce model overfitting [
      • Friedman J.
      • Hastie T.
      • Tibshirani R
      Regularization paths for generalized linear models via coordinate descent.
      ]. LASSO added the L1 norm of the feature coefficients as a penalty term to the loss function, which forced the coefficients corresponding to those weak variables to become zero. Herein, we considered variables whose coefficients were equal to zero as redundant variables, and all of the variables with nonzero coefficients were retained for subsequent analysis [
      • Gao Y.
      • Cai G.-.Y.
      • Fang W.
      • Li H.-.Y.
      • Wang S.-.Y.
      • Chen L.
      • et al.
      Machine learning based early warning system enables accurate mortality risk prediction for COVID-19.
      ]. Details on the variable extraction and selection are provided in the Supplementary Methods section.

      2.3 Prognostic model derivation

      The multivariate logistic regression model was established based on the variables selected by the LASSO regression in the training set. To provide the clinician with a quantitative tool to predict the outcomes of patients with ALC, we next built a web-based Nomogram for Intensive care unit patients with ALC (NIALC) through the R package “DynNom”. The discriminant function of the nomogram was evaluated by plotting the receiver operating curve (ROC) and calculating the area under the curve (AUC). Calibration curves were plotted to assess the calibration of the nomogram, accompanied by the Hosmer–Lemeshow test. Decision curve analysis (DCA) was conducted to determine the clinical usefulness of the nomogram by quantifying the net benefits at different threshold probabilities in the validation dataset.

      2.4 Model validation

      To validate our model, three criteria were used to evaluate prediction performance in the internal and external validation cohorts. First, the cases were grouped according to their predicted risk score, and Kaplan–Meier survival curves and the Wald test were used to compare the survival differences among the groups. Second, given that many prognostic scores for severe liver diseases were produced, we compared our model with the four selected scores that were widely used and exhibited good performance in previous studies, including the MELD, MELD-sodium, Child–Pugh, and CLIF-SOFA scores [
      • Wu T.
      • Li J.
      • Shao L.
      • Xin J.
      • Jiang L.
      • Zhou Q.
      • et al.
      Development of diagnostic criteria and a prognostic score for hepatitis B virus-related acute-on-chronic liver failure.
      ]. The MELD score was calculated using the formula: MELD=3.78×ln[Totalbilirubin(mg/dL)]+11.2×ln(InternationalNormalizedRatio)+9.57×ln[Serumcreatinine(mg/dL)] [
      • Kamath P.S.
      • Wiesner R.H.
      • Malinchoc M.
      • Kremers W.
      • Therneau T.M.
      • Kosberg C.L.
      • et al.
      A model to predict survival in patients with end-stage liver disease.
      ]. The MELD-sodium score was calculated using the formula: MELDsodium=MELDserumsodium[0.025×MELD×(140serumsodium)]+140 [
      • Biggins S.W.
      • Kim W.R.
      • Terrault N.A.
      • Saab S.
      • Balan V.
      • Schiano T.
      • et al.
      Evidence-based incorporation of serum sodium concentration into MELD.
      ]. The Child–Pugh score and CLIF-SOFA score were also calculated [
      • Moreau R.
      • Jalan R.
      • Gines P.
      • Pavesi M.
      • Angeli P.
      • Cordoba J.
      • et al.
      Acute-on-chronic liver failure is a distinct syndrome that develops in patients with acute decompensation of cirrhosis.
      ,
      • Child C.G.
      • Turcotte J.G
      Surgery and portal hypertension.
      . To compare the predictive accuracy between our model and the other four prognostic scores, we assessed and compared their discrimination using AUC. Third, we also validated our model in alcoholic cirrhotic patients with different complications, including GIB, AKI, ascites, hepatic encephalopathy, and spontaneous bacterial peritonitis.

      2.5 Statistical analysis

      Descriptive analyses of the patients were conducted. Continuous variables are expressed as the mean and standard deviation (SD) or median and interquartile range (IQR), as appropriate, while categorical variables are presented as frequencies and percentages. To compare baseline characteristics between the cohorts, we used either Student's t-test or the Mann–Whitney U test. The distribution of categorical variables was compared using the chi-square and Fisher's exact tests, as appropriate. The results are considered significant with p less than 0.05. All analyses were computed using Python version 3.9 (Python Software Foundation) and R version 4.0.1 (R Foundation for Statistical Computing).

      3. Results

      3.1 Characteristics of the study populations

      Fig. 1 depicts the flowchart diagram of patient selection. Among 883 and 614 patients with ALC from the MIMIC and eICU-CRD databases, 788 and 501 were finally included in the analyses, respectively. The clinical characteristics of all included patients are summarized in the Table 1. The derivation and external validation cohorts differed in their demographics, routine blood tests, and liver and kidney functions. In the derivation cohort, AKI was the most frequent complication of cirrhosis (231 [58.6%]), while gastrointestinal bleeding was the most frequent complication of cirrhosis (231 [46.1%]) in the external validation cohort. Mortality in the derivation cohort was as high as 41.4%, whereas only 104 (20.8%) patients with ALC died during the ICU stay in the external validation cohort.
      Table 1Patient characteristics.
      Patient characteristicsDerivation cohort (n = 394)Internal validation cohort (n = 394)External validation cohort (n = 501)p value
      Demographics
       Age, years55.6 (10.3)56.2 (15.6)54.2 (10.7)0.043
       Sex, male (n,%)

      Vital signs
      278 (70.6%)293 (74.4%)342 (68.3%)0.506
       Heart rate, per minute90.09 (15.02)88.29 (14.65)89.63 (15.80)0.659
       MAP, mmHg79.99 (10.30)79.34 (10.81)78.99 (12.57)0.191
      Liver and kidney function
       ALT, Units/L66.92 (26.25 −125.29)71.00 (28.62 −125.29)52.00 (28.00 −68.76)<0.001
       AST, Units/L133.00 (56.62 −281.91)157.50 (59.25 −281.91)112.00 (55.00 −204.42)0.145
       Albumin, g/dL2.89 (2.80 −2.95)2.89 (2.83 −2.90)2.52 (2.20 −2.70)<0.001
       Total bilirubin, mg/dL6.78 (2.80 −6.78)6.78 (2.45 −6.78)6.60 (2.50 −6.61)0.726
       BUN, mg/dL27.42 (15.78 −33.00)27.00 (15.00 −33.25)23.00 (13.00 −30.83)0.026
       Serum creatinine, mg/dL1.10 (1.00 −2.10)1.14 (0.70 −1.40)1.07 (0.73 −1.46)0.939
      Routine blood test
       Hgb, g/dL10.08 (9.05 −10.68)10.08 (9.28 −10.80)9.47 (8.20 −10.40)<0.001
       Hct,%29.50 (26.65 −30.99)29.50 (27.21 −31.18)28.14 (24.24 −30.90)<0.001
       MCH, pg32.45 (30.69 −33.50)32.45 (31.00 −34.05)31.91 (30.16 −33.10)0.109
       MCHC, g/dL34.19 (33.43 −34.88)34.19 (33.60 −35.00)33.65 (33.00 −34.50)<0.001
      Blood coagulation test
       PT, second18.33 (15.80 −18.90)18.06 (15.80 −19.07)20.62 (17.55 −20.62)<0.001
       PTT, second40.10 (33.82 −42.10)41.24 (34.04 −43.91)40.94 (40.80 −40.94)0.285
       INR1.70 (1.40 −1.90)1.70 (1.40 −2.00)1.95 (1.50 −1.95)0.210
      Comorbidities
       Hypertension (n,%)90 (22.8%)72 (18.3%)47 (9.4%)<0.001
       Diabetes (n,%)6 (1.5%)6 (1.5%)45 (9.0%)<0.001
      Complications
       GIB (n,%)71 (18.0%)85 (21.6%)231 (46.1%)<0.001
       AKI (n,%)231 (58.6%)220 (55.8%)142 (28.3%)<0.001
       Ascites (n,%)114 (28.9%)113 (28.7%)91 (18.2%)<0.001
       HE (n,%)141 (35.8%)136 (34.5%)97 (19.4%)<0.001
       SBP (n,%)34 (8.6%)45 (11.4%)24 (4.8%)0.029
      Severity score
       MELD22.19 (15.51 −29.48)20.62 (14.73 −26.85)21.57 (15.85 −30.91)0.680
       MELD-sodium23.81 (16.23 −32.37)22.34 (15.70 −29.77)24.45 (16.99 −31.74)0.390
       Child-Pugh8.00 (6.00 −10.00)8.00 (6.00 −10.00)7.00 (6.00 −9.00)0.003
       CLIF-SOFA6.00 (4.00 −9.00)6.00 (4.00 −8.00)7.00 (5.00 −10.00)0.001
      Outcomes
       Length of stay, days10 (5–20)10 (5–18)4 (2–7)<0.001
       Death (n,%)163 (41.4%)169 (42.9%)104 (20.8%)<0.001
      Continuous variables are expressed as the mean and standard deviation (SD) or median and interquartile range (IQR), whereas categorical variables are reported as n (%). The independent t-test (or Mann–Whitney U test for non-parametric distributions) and chi-square test were used to compare continuous and categorical characteristics between patients in derivation and external validation cohorts.
      Abbreviations: MAP, Mean Arterial Pressure; ALT, Alanine Aminotransferase; AST, Aspartate Transaminase; BUN, Blood Urea Nitrogen; SCr, Serum Creatinine; Hgb, Hemoglobin; Hct, Hematocrit; MCH, Mean Corpuscular Hemoglobin; MCHC, Mean Corpuscular Hemoglobin Concentration; PT, Prothrombin Time; PTT, Partial Thromboplastin Time; INR, International Normalized Ratio; GIB, Gastrointestinal Bleeding; AKI, Acute Kidney Injury; HE, Hepatic Encephalopathy; SBP, Systolic Blood Pressure; MELD, Model of End-Stage Liver Disease; CLIF-SOFA, Chronic Liver Failure-Sequential Organ Failure Assessment score.

      3.2 Variable selection and model derivation

      Among the 73 variables considered in this analysis (Supplementary Figure 1), those variables with missing values greater than 90% were excluded, resulting in 52 variables. Then, Pearson's correlation was used to determine the correlation matrix between 52 variables, and only one variable was retained when removing pairs of variables with coefficients greater than 0.8 to avoid collinearity, resulting in 46 variables. Fig. 2 shows that only 11 of the 46 variables were eventually chosen for model development by LASSO regression, among which 7 variables had a positive association with in-hospital mortality (including blood urea nitrogen [BUN], total bilirubin [Tbil], prothrombin time [PT], serum creatinine [SCr], lactate, partial thromboplastin time [PTT], and phosphate), and 4 variables were negatively correlated with in-hospital mortality (mean arterial pressure [MAP], lymphocytes, fibrinogen, and albumin).
      Fig. 2
      Fig. 2Feature selection by LASSO. A LASSO coefficient profiles of the 73 variables. A coefficient profile plot was produced against the log Lambda sequence. B The vertical dashed line was drawn at the value selected using 10-fold cross-validation, where optimal lambda resulted in 11 nonzero coefficients. Abbreviations: BUN, blood urea nitrogen; SCr, serum creatinine; TBil, total bilirubin; SaO2, oxygen saturation; PT, prothrombin time; PTT, partial thromboplastin time; RDW, red blood cell distribution width; MCV, mean corpuscular volume; MCH, mean corpuscular hemoglobin; MCHC, mean corpuscular hemoglobin concentration; CPK, creatinine phosphokinase; AST, aspartate transaminase; ALT, alanine aminotransferase; AKP, alkaline phosphatase; INR, international normalized ratio; PaO2, partial pressure of oxygen; FiO2, fraction of inspiration O2; PaCO2, partial pressure of carbon dioxide; MAP, mean arterial pressure.
      The results of the multivariate binary logistic regression model are shown in the Supplementary Table 1. Based on 11 variables, the risk score was calculated by adding up the score of each item using the nomogram depicted in Fig. 3. Lactate showed the largest range of risk scores, followed by SCr and PTT. The corresponding predicted risk of mortality for each risk sore was presented in Supplementary Figure 2. An online version of our nomogram can be accessed at https://zhengnomogram.shinyapps.io/NIALC/ to assist researchers and clinicians. The predicted survival probability during the ICU stay can be easily determined by inputting the clinical variables and reading the output figures and tables generated by the webserver.
      Fig. 3
      Fig. 3Nomogram for ICU patients with ALC (NIALC). Abbreviations: ALC, alcohol-associated cirrhosis; BUN, blood urea nitrogen; PTT, partial thromboplastin time; PT, prothrombin time; MAP, mean arterial pressure.

      3.3 Model validation

      The proposed nomogram was validated in the internal (n = 394) and external (n = 501) validation cohorts. Supplementary Figure 3 shows the performance of NIALC. Our nomogram showed good discrimination ability with AUCs of 0.767 (95% CI, 0.720 to 0.815) and 0.760 (95% CI, 0.708 to 0.811) in the internal and external validation cohorts, respectively (Supplementary Figure 3A and 3D). The calibration curves for the probability of mortality in the internal and external validation cohorts demonstrated good agreement between the prediction and observation (Supplementary Figure 3B and 3E). The Hosmer–Lemeshow test indicated that the proposed nomogram was well-calibrated in the internal (p = 0.920) and external (p = 0.895) validation sets. Moreover, DCA was also performed to render the nomogram clinically useful. The decision curves showed that clinical intervention guided by this nomogram provided a greater net benefit when the threshold probability ranged from 20% to 90% (Supplementary Figure 3C and 3F). Moreover, we divided the patients into 2 risk groups based on their predicted scores. Patients in the high-risk group had a significantly (P < 0.05) higher risk of death than patients in the low-risk group (Supplementary Figure 4).

      3.4 Subgroup analysis

      Fig. 4 depicts the comparisons of NIALC and previous prognostic scores. In the internal validation cohort, the proposed nomogram performed better than the four commonly used prognostic scores. In the external validation cohort, the proposed nomogram performed better than the Child–Pugh and CLIF-SOFA. Supplementary Figure 5 shows the performance of NIALC and the previous models in different subgroups. In the internal validation cohort, NIALC retained the best and most stable performance across all groups (patients with diabetes were excluded because there were so few of them). In the external validation cohort, the NIALC and MELD scores were comparable across all groups.
      Fig. 4
      Fig. 4Comparison of the new model's predictive performance with previous models. A ROC curves for the internal validation cohort. B ROC curves for the external validation cohort. C AUC of the five models’ predictive performance. The table summarizes the results with 95% confidence intervals. Abbreviations: NIALC, Nomogram for Intensive Care Unit Patients with Alcohol-Associated Cirrhosis; MELD, Model of End-Stage Liver Disease; CLIF-SOFA, Chronic Liver Failure-Sequential Organ Failure Assessment score.

      4. Discussion

      In the present study, we developed a nomogram (NIALC) for ICU patients with ALC. NIALC consisted of 11 baseline parameters, including BUN, Tbil, PT, SCr, lactate, PTT, phosphate, MAP, lymphocytes, fibrinogen, and albumin. The NIALC model showed satisfactory discriminant function (AUC, 0.767; 95% CI, 0.720 to 0.815) and performed better than other commonly used models (MELD, MELD-sodium, CTP, and CLIF-SOFA) for ICU patients with ALC. We also developed an online calculator (https://zhengnomogram.shinyapps.io/NIALC/) to allow accurate mortality risk stratification for ICU patients with ALC, which may facilitate a rapid response that benefits high-risk patients through early identification, enhanced care and monitoring and subsequent immediate intervention.
      Early prediction of the outcome for ICU patients with DC is of great importance, as it can be used to guide clinical management and decrease mortality [
      • Ginès P.
      • Fernández J.
      • Durand F.
      • Saliba F
      Management of critically-ill cirrhotic patients.
      ]. Although several prognostic scores have been developed for ICU patients with cirrhosis, they have not gained widespread acceptance primarily because of their potential limitations mentioned in the introduction [
      • Cholongitas E.
      • Senzolo M.
      • Patch D.
      • Kwong K.
      • Nikolopoulou V.
      • Leandro G.
      • et al.
      Risk factors, sequential organ failure assessment and model for end-stage liver disease scores for predicting short term mortality in cirrhotic patients admitted to intensive care unit.
      ,
      • Fang J.-.T.
      • Tsai M.-.H.
      • Tian Y.-.C.
      • Jenq C.-.C.
      • Lin C.-.Y.
      • Chen Y.-.C.
      • et al.
      Outcome predictors and new score of critically ill cirrhotic patients with acute renal failure.
      ,
      • Zauner C.
      • Schneeweiss B.
      • Schneider B.
      • Madl C.
      • Klos H.
      • Kranz A.
      • et al.
      Short-term prognosis in critically ill patients with liver cirrhosis: an evaluation of a new scoring system.
      ,
      • Theocharidou E.
      • Pieri G.
      • Mohammad A.O.
      • Cheung M.
      • Cholongitas E.
      • Agarwal B.
      • et al.
      The Royal Free Hospital score: a calibrated prognostic model for patients with cirrhosis admitted to intensive care unit. Comparison with current models and CLIF-SOFA score.
      ]. In addition, the various etiologies of liver cirrhosis were not considered in previous studies, and the epidemiology, histological features, and clinical symptoms varied between cirrhosis of alcoholic and nonalcoholic etiology [
      • Lackner C.
      • Tiniakos D
      Fibrosis and alcohol-related liver disease.
      ]. Therefore, it is necessary to develop a specific prognostic score for ICU patients with ALC. The ICU is a highly challenging environment that confronts physicians with a demanding case load requiring rapid decision-making. Our results also indicated that the NIALC was easier to use and performed better than previous models in ICU patients with ALC.
      The parameters in the NIALC could be grouped into 4 categories: 1) coagulation measures (PT, PTT, and fibrinogen), 2) measures of renal function (BUN, SCr, and phosphate), 3) measures of liver function (Tbil, albumin, and lymphocytes), and 4) hemodynamic index (MAP and lactate). In fact, previous prognostic scores have demonstrated that increased mortality was associated with elevated kidney function tests (BUN and SCr), abnormal liver function (elevated Tbil and decreased albumin), coagulation panel (elevated PT and PTT), and hemodynamic index (MAP) [
      • Wu T.
      • Li J.
      • Shao L.
      • Xin J.
      • Jiang L.
      • Zhou Q.
      • et al.
      Development of diagnostic criteria and a prognostic score for hepatitis B virus-related acute-on-chronic liver failure.
      ,
      • Kamath P.S.
      • Wiesner R.H.
      • Malinchoc M.
      • Kremers W.
      • Therneau T.M.
      • Kosberg C.L.
      • et al.
      A model to predict survival in patients with end-stage liver disease.
      ,
      • Moreau R.
      • Jalan R.
      • Gines P.
      • Pavesi M.
      • Angeli P.
      • Cordoba J.
      • et al.
      Acute-on-chronic liver failure is a distinct syndrome that develops in patients with acute decompensation of cirrhosis.
      ,
      • Child C.G.
      • Turcotte J.G
      Surgery and portal hypertension.
      ,
      • Deshmukh F.
      • Merchant S.S
      Explainable machine learning model for predicting GI bleed mortality in the intensive care unit.
      . In addition, fibrinogen, phosphate, lymphocytes, and lactate have also been found to be important prognostic factors in NIALC, which were not included in previous prognostic scores.
      Lactate reflects the balance between tissue oxygen demand and blood oxygen supply capacity. A previous study of lactate elimination between healthy volunteers and cirrhotic patients showed a significant prolongation of lactate half-life in cirrhotic patients [
      • Woll P.J.
      • Record C.O
      Lactate elimination in man: effects of lactate concentration and hepatic dysfunction.
      ]. This suggested that hepatic lactate uptake was impaired in cirrhotic patients due either to hepatocyte dysfunction or portal diversion [
      • Woll P.J.
      • Record C.O
      Lactate elimination in man: effects of lactate concentration and hepatic dysfunction.
      ]. The primary physiological function of serum phosphate is to help maintain the blood acid-base balance, which is mainly filtered by glomeruli and reabsorbed by renal tubules. Patients with ALC tend to have splanchnic and systemic vasodilatation caused by portal hypertension, which leads to activation of the renin-angiotensin-aldosterone system (RAAS) [
      • Schrier R.W.
      • Arroyo V.
      • Bernardi M.
      • Epstein M.
      • Henriksen J.H.
      • Rodés J
      Peripheral arterial vasodilation hypothesis: a proposal for the initiation of renal sodium and water retention in cirrhosis.
      ]. RAAS further contributes to a decrease in renal blood flow and renal failure, which then decreases urinary phosphorus excretion [
      • Ring-Larsen H
      Renal blood flow in cirrhosis: relation to systemic and portal haemodynamics and liver function.
      ]. Therefore, elevated serum phosphate and lactate could be strong predictors of the outcome in patients with ALC.
      In addition, fibrinogen is a plasma glycoprotein synthesized by the liver and is the main protein involved in clot formation and wound healing [
      • Gligorijević N.
      • Minić S.
      • Križáková M.
      • Katrlík J.
      • Nedić O
      Structural changes of fibrinogen as a consequence of cirrhosis.
      ]. In the decompensation stage, loss of hepatocytes with normal function leads to a decline in protein synthesis, including fibrinogen, albumin, and immunoglobulins [
      • Shacter E.
      • Williams J.A.
      • Lim M.
      • Levine R.L
      Differential susceptibility of plasma proteins to oxidative modification: examination by western blot immunoassay.
      ]. Lymphocytes play a crucial role in the induction and regulation of immune responses. Because of a universal defect in thymopoiesis exacerbated by splenic pooling and activation-driven cell death induced by bacterial translocation, patients with cirrhosis always show reduced numbers of T-helper lymphocytes [
      • Lario M.
      • Muñoz L.
      • Ubeda M.
      • Borrero M.-.J.
      • Martínez J.
      • Monserrat J.
      • et al.
      Defective thymopoiesis and poor peripheral homeostatic replenishment of T-helper cells cause T-cell lymphopenia in cirrhosis.
      ]. In patients with alcohol-related liver disease, the decline in the percentage of lymphocytes appears more obvious, and this alteration is exacerbated with worsening disease severity [
      • Albillos A.
      • Martin-Mateos R.
      • Van der Merwe S.
      • Wiest R.
      • Jalan R.
      • Álvarez-Mon M
      Cirrhosis-associated immune dysfunction.
      ]. Consistent with prior studies, our study also showed that a decline in fibrinogen and the percentage of blood lymphocytes were associated with a worse prognosis in patients with ALC.

      4.1 Strengths and limitations

      To our knowledge, this is the first study to include a large sample size that developed and validated a prognostic model for ICU patients with ALC. In this simple model, the most important prognostic variables were selected from a large number of candidate variables using a machine learning approach. Moreover, our model showed good performance in both the internal and external validation cohorts with heterogeneous baseline characteristics. The subgroup analyses also showed that the prognostic model was robust. In addition to the abovementioned strengths, there were also limitations to the study. First, the model was built based on retrospective data, which were collected and stored in an electronic database. Because prognostic factors in our study are recorded before the occurrence of the outcome, bias inherent in traditionally retrospective data has been minimized here. Second, some candidate prognostic factors, including ammonia and C-reactive protein, were excluded during model development because of a large proportion of missing values in the database. The inclusion of those variables might further improve the predictive capabilities of our nomogram. Third, the study participants in the derivation and validation cohorts were exclusively enrolled in the United States, which may limit the generalization of this model. Lastly, the primary outcome is in hospital mortality, and the accuracy of our model in predicting long-term outcomes is unknown and warrant further evaluation.

      5. Conclusions

      In summary, this study developed a nomogram for predicting death in ICU patients with ALC and externally validated the utility of the nomogram. The nomogram showed satisfactory performance. This easily applicable nomogram uses routinely collected clinical variables and could be accessed online, which may facilitate optimal prognostic management of ICU patients with ALC based on their score-based risk stratification.

      Conflict of interest

      None declared.

      Acknowledgments

      None.

      Data Availability Statement

      The data from the Medical Information Mart for Intensive Care (MIMIC) and eICU-CRD databases were available on website at https://physionet.org/about/database/.

      Funding

      This study was funded by the National Natural Science Foundation of China (71904170), Mega-Project of National Science and Technology for the 13th Five-Year Plan of China (2018ZX10721102-003-006, 2018ZX10715013-003-003), Technological Innovation Leading Talents of “Ten Thousand Talents Plan” of Zhejiang Province (2020R52010), and the Fundamental Research Funds for the Central Universities (2022ZFJH003, K20210205).

      Author contributions

      M.Z. and J.W. contributed to the study equally. M.Z. and J.W. conceived and designed the study. L.Z. acquired the data and performed analyses assisted by J.W. and M.Z. L.Z. and Y.L. drafted the manuscript and all authors critically revised it. M.Z. is the guarantor. All authors were responsible for interpretation of data and for approving the draft manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

      Appendix. Supplementary materials

      References

        • Rehm J.
        • Samokhvalov A.V.
        • Shield K.D
        Global burden of alcoholic liver diseases.
        J Hepatol. 2013; 59: 160-168https://doi.org/10.1016/j.jhep.2013.03.007
        • Seitz H.K.
        • Bataller R.
        • Cortez-Pinto H.
        • Gao B.
        • Gual A.
        • Lackner C.
        • et al.
        Alcoholic liver disease.
        Nat Rev Dis Prim. 2018; 4: 16https://doi.org/10.1038/s41572-018-0014-7
        • Kevin D
        • Shield M.R.
        • R J.
        Public health successes and missed opportunities.
        Trends in alcohol consumption and attributable mortality in the WHO European Region, 1990–2014. 2016
        • Cholongitas E.
        • Senzolo M.
        • Patch D.
        • Shaw S.
        • Hui C.
        • Burroughs A.K
        Review article: scoring systems for assessing prognosis in critically ill adult cirrhotics.
        Aliment Pharmacol Ther. 2006; 24: 453-464https://doi.org/10.1111/j.1365-2036.2006.02998.x
        • Ginès P.
        • Fernández J.
        • Durand F.
        • Saliba F
        Management of critically-ill cirrhotic patients.
        J Hepatol. 2012; 56 Suppl 1: S13-S24https://doi.org/10.1016/S0168-8278(12)60003-8
        • Cholongitas E.
        • Senzolo M.
        • Patch D.
        • Kwong K.
        • Nikolopoulou V.
        • Leandro G.
        • et al.
        Risk factors, sequential organ failure assessment and model for end-stage liver disease scores for predicting short term mortality in cirrhotic patients admitted to intensive care unit.
        Aliment Pharmacol Ther. 2006; 23: 883-893https://doi.org/10.1111/j.1365-2036.2006.02842.x
        • Fang J.-.T.
        • Tsai M.-.H.
        • Tian Y.-.C.
        • Jenq C.-.C.
        • Lin C.-.Y.
        • Chen Y.-.C.
        • et al.
        Outcome predictors and new score of critically ill cirrhotic patients with acute renal failure.
        Nephrol Dial Transplant. 2008; 23: 1961-1969https://doi.org/10.1093/ndt/gfm914
        • Zauner C.
        • Schneeweiss B.
        • Schneider B.
        • Madl C.
        • Klos H.
        • Kranz A.
        • et al.
        Short-term prognosis in critically ill patients with liver cirrhosis: an evaluation of a new scoring system.
        Eur J Gastroenterol Hepatol. 2000; 12: 517-522https://doi.org/10.1097/00042737-200012050-00007
        • Theocharidou E.
        • Pieri G.
        • Mohammad A.O.
        • Cheung M.
        • Cholongitas E.
        • Agarwal B.
        • et al.
        The Royal Free Hospital score: a calibrated prognostic model for patients with cirrhosis admitted to intensive care unit. Comparison with current models and CLIF-SOFA score.
        Am J Gastroenterol. 2014; 109: 554-562https://doi.org/10.1038/ajg.2013.466
        • Biggins S.W.
        • Kim W.R.
        • Terrault N.A.
        • Saab S.
        • Balan V.
        • Schiano T.
        • et al.
        Evidence-based incorporation of serum sodium concentration into MELD.
        Gastroenterology. 2006; 130: 1652-1660https://doi.org/10.1053/j.gastro.2006.02.010
        • Kim H.Y.
        • Kim C.W.
        • Kim T.Y.
        • Song D.S.
        • Sinn D.H.
        • Yoon E.L.
        • et al.
        Assessment of scoring systems for acute-on-chronic liver failure at predicting short-term mortality in patients with alcoholic hepatitis.
        World J Gastroenterol. 2016; 22: 9205-9213https://doi.org/10.3748/wjg.v22.i41.9205
        • Johnson A.E.W.
        • Pollard T.J.
        • Shen L.
        • Lehman L-WH
        • Feng M.
        • Ghassemi M.
        • et al.
        MIMIC-III, a freely accessible critical care database.
        Sci Data. 2016; 3160035https://doi.org/10.1038/sdata.2016.35
        • Pollard T.J.
        • Johnson A.E.W.
        • Raffa J.D.
        • Celi L.A.
        • Mark R.G.
        • Badawi O
        The eICU Collaborative Research Database, a freely available multi-center database for critical care research.
        Sci Data. 2018; 5180178https://doi.org/10.1038/sdata.2018.178
        • Moons K.G.M.
        • Altman D.G.
        • Reitsma J.B.
        • Ioannidis J.P.A.
        • Macaskill P.
        • Steyerberg E.W.
        • et al.
        Transparent reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration.
        Ann Intern Med. 2015; 162: W1-73https://doi.org/10.7326/M14-0698
        • Friedman J.
        • Hastie T.
        • Tibshirani R
        Regularization paths for generalized linear models via coordinate descent.
        J Stat Softw. 2010; 33: 1-22
        • Gao Y.
        • Cai G.-.Y.
        • Fang W.
        • Li H.-.Y.
        • Wang S.-.Y.
        • Chen L.
        • et al.
        Machine learning based early warning system enables accurate mortality risk prediction for COVID-19.
        Nat Commun. 2020; 11: 5033https://doi.org/10.1038/s41467-020-18684-2
        • Wu T.
        • Li J.
        • Shao L.
        • Xin J.
        • Jiang L.
        • Zhou Q.
        • et al.
        Development of diagnostic criteria and a prognostic score for hepatitis B virus-related acute-on-chronic liver failure.
        Gut. 2018; 67: 2181-2191https://doi.org/10.1136/gutjnl-2017-314641
        • Kamath P.S.
        • Wiesner R.H.
        • Malinchoc M.
        • Kremers W.
        • Therneau T.M.
        • Kosberg C.L.
        • et al.
        A model to predict survival in patients with end-stage liver disease.
        Hepatology. 2001; 33: 464-470https://doi.org/10.1053/jhep.2001.22172
        • Moreau R.
        • Jalan R.
        • Gines P.
        • Pavesi M.
        • Angeli P.
        • Cordoba J.
        • et al.
        Acute-on-chronic liver failure is a distinct syndrome that develops in patients with acute decompensation of cirrhosis.
        Gastroenterology. 2013; 144 (1437.e1-9): 1426-1437https://doi.org/10.1053/j.gastro.2013.02.042
        • Child C.G.
        • Turcotte J.G
        Surgery and portal hypertension.
        Major Probl Clin Surg. 1964; 1: 1-85
        • Lackner C.
        • Tiniakos D
        Fibrosis and alcohol-related liver disease.
        J Hepatol. 2019; 70: 294-304https://doi.org/10.1016/j.jhep.2018.12.003
        • Deshmukh F.
        • Merchant S.S
        Explainable machine learning model for predicting GI bleed mortality in the intensive care unit.
        Am J Gastroenterol. 2020; 115: 1657-1668https://doi.org/10.14309/ajg.0000000000000632
        • Woll P.J.
        • Record C.O
        Lactate elimination in man: effects of lactate concentration and hepatic dysfunction.
        Eur J Clin Invest. 1979; 9: 397-404https://doi.org/10.1111/j.1365-2362.1979.tb00903.x
        • Schrier R.W.
        • Arroyo V.
        • Bernardi M.
        • Epstein M.
        • Henriksen J.H.
        • Rodés J
        Peripheral arterial vasodilation hypothesis: a proposal for the initiation of renal sodium and water retention in cirrhosis.
        Hepatology. 1988; 8: 1151-1157https://doi.org/10.1002/hep.1840080532
        • Ring-Larsen H
        Renal blood flow in cirrhosis: relation to systemic and portal haemodynamics and liver function.
        Scand J Clin Lab Invest. 1977; 37: 635-642https://doi.org/10.3109/00365517709100657
        • Gligorijević N.
        • Minić S.
        • Križáková M.
        • Katrlík J.
        • Nedić O
        Structural changes of fibrinogen as a consequence of cirrhosis.
        Thromb Res. 2018; 166: 43-49https://doi.org/10.1016/j.thromres.2018.04.005
        • Shacter E.
        • Williams J.A.
        • Lim M.
        • Levine R.L
        Differential susceptibility of plasma proteins to oxidative modification: examination by western blot immunoassay.
        Free Radic Biol Med. 1994; 17: 429-437https://doi.org/10.1016/0891-5849(94)90169-4
        • Lario M.
        • Muñoz L.
        • Ubeda M.
        • Borrero M.-.J.
        • Martínez J.
        • Monserrat J.
        • et al.
        Defective thymopoiesis and poor peripheral homeostatic replenishment of T-helper cells cause T-cell lymphopenia in cirrhosis.
        J Hepatol. 2013; 59: 723-730https://doi.org/10.1016/j.jhep.2013.05.042
        • Albillos A.
        • Martin-Mateos R.
        • Van der Merwe S.
        • Wiest R.
        • Jalan R.
        • Álvarez-Mon M
        Cirrhosis-associated immune dysfunction.
        Nat Rev Gastroenterol Hepatol. 2022; 19: 112-134https://doi.org/10.1038/s41575-021-00520-7