Research Article| Volume 49, ISSUE 4, P407-414, July 2020

Download started.


Is my patient ready for a safe transfer to a lower-intensity care setting? Nursing complexity as an independent predictor of adverse events risk after ICU discharge

Published:February 14, 2020DOI:


      • A residual clinical instability and functional deficit is fully expected for patients discharge from ICU.
      • Patients at high risk for adverse events can be effectively identified before ICU discharge based on routinely collected data.
      • The level of acuity and nursing complexity can independently predict the risk of adverse events.
      • Using such information to support the discharge decision-making could prevent adverse events and improve patient's outcome.



      Patients discharged from intensive care units (ICUs) are at risk for adverse events (AEs). Establishing safe discharge criteria is challenging. No available criteria consider nursing complexity among risk factors.


      To investigate whether nursing complexity upon ICU discharge is an independent predictor for AEs.


      Prospective observational study. The Patient Acuity and Complexity Score (PACS) was developed to measure nursing complexity. Its predictive power for AEs was tested using multivariate regression analysis.


      The final regression model showed a very-good discrimination power (AUC 0.881; p<0.001) for identifying patients who experienced AEs. Age, ICU admission reason, PACS, cough strength, PaCO2, serum creatinine and sodium, and transfer to Internal Medicine showed to be predictive of AEs. Exceeding the identified PACS threshold increased by 3.3 times the AEs risk.


      The level of nursing complexity independently predicts AEs risk and should be considered in establishing patient's eligibility for a safe ICU discharge.



      ADL (Activities of Daily Living), APACHE II (Acute Physiology And Chronic Health Evaluation), ATP (adenosine triphosphate), AUC (area under receiver operating characteristics curve), AVPU (alert, verbal, pain, unresponsive), CI (confidence interval), CVI (content validity index), FiO2 (fraction of inspired oxygen), ICU (intensive care units), IQR (interquartile range), NEWS (National Early Warning Score), OR (odds ratio), PaCO2 (partial pressure of arterial carbon dioxide), PACS (Patient Acuity and Complexity Score), PaO2 (partial pressure of oxygen)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Heart & Lung: The Journal of Cardiopulmonary and Acute Care
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Nates J.L.
        • Nunnally M.
        • Kleinpell R.
        • et al.
        ICU admission, discharge, and triage guidelines: a framework to enhance clinical operations, development of institutional policies, and further research.
        Crit Care Med. 2016; 44: 1553-1602
        • Bench S.
        • Day T.
        The user experience of critical care discharge: a meta-synthesis of qualitative research.
        Int J Nurs Stud. 2010; 47: 487-499
        • Li P.
        • Stelfox H.T.
        • Ghali W.A
        A prospective observational study of physician handoff for intensive-care-unit-to-ward patient transfers.
        Am J Med. 2011; 124: 860-867
        • World Health Organization
        The Conceptual Framework for the International Classification for Patient Safety (v1. 1) Final Technical Report and Technical Annexes.
        World Health Organization Global Press, 2009
        • Brennan T.A.
        • Leape L.L.
        • Laird N.M.
        • et al.
        Incidence of adverse events and negligence in hospitalized patients: results of the Harvard Medical Practice study I. 1991.
        Qual Saf Health Care. 2004; 13: 145-151
        • Wilson R.M.
        • Runciman W.B.
        • Gibberd R.W.
        • Harrison B.T.
        • Newby L.
        • Hamilton J.D
        The quality in Australian health care study.
        Med J Aust. 1995; 163: 458-471
        • Ng Y.H.
        • Pilcher D.V.
        • Bailey M.
        • Bain C.A.
        • MacManus C.
        • Bucknall T.K
        Predicting medical emergency team calls, cardiac arrest calls and re-admission after intensive care discharge: creation of a tool to identify at-risk patients.
        Anaesth Intensiv Care. 2018; 46: 88-96
        • Frick S.
        • Uehlinger D.E.
        • Zuercher Zenklusen R.M
        Medical futility: predicting outcome of intensive care unit patients by nurses and doctors–a prospective comparative study.
        Crit Care Med. 2003; 31: 456-461
        • Andersen L.W.
        • Kim W.Y.
        • Chase M.
        • et al.
        The prevalence and significance of abnormal vital signs prior to in-hospital cardiac arrest.
        Resuscitation. 2016; 98: 112-117
        • Berlot G.
        • Pangher A.
        • Petrucci L.
        • Bussani R.
        • Lucangelo U
        Anticipating events of in-hospital cardiac arrest.
        Eur J Emerg Med. 2004; 11: 24-28
        • McLaughlin N.
        • Leslie G.D.
        • Williams T.A.
        • Dobb G.J
        Examining the occurrence of adverse events within 72h of discharge from the intensive care unit.
        Anaesth Intensiv Care. 2007; 35: 486-493
        • McQuillan P.
        • Pilkington S.
        • Allan A.
        • et al.
        Confidential inquiry into quality of care before admission to intensive care.
        BMJ. 1998; 316: 1853-1858
        • Hodgetts T.J.
        • Kenward G.
        • Vlackonikolis I.
        • et al.
        Incidence, location and reasons for avoidable in-hospital cardiac arrest in a district general hospital.
        Resuscitation. 2002; 54: 115-123
        • Needleman J.
        • Buerhaus P.
        • Pankratz V.S.
        • Leibson C.L.
        • Stevens S.R.
        • Harris M
        Nurse staffing and inpatient hospital mortality.
        N Engl J Med. 2011; 364: 1037-1045
        • Campbell A.J.
        • Cook J.A.
        • Adey G.
        • Cuthbertson B.H
        Predicting death and readmission after intensive care discharge.
        Br J Anaesth. 2008; 100: 656-662
        • Chaboyer W.
        • Thalib L.
        • Foster M.
        • Ball C.
        • Richards B
        Predictors of adverse events in patients after discharge from the intensive care unit.
        Am J Crit Care. 2008; 17: 255-263
        • McGloin H.
        • Adam S.K.
        • Singer M
        Unexpected deaths and referrals to intensive care of patients on general wards. Are some cases potentially avoidable?.
        J R Coll Physicians Lond. 1999; 33: 255-259
        • Elliott M.
        • Worrall-Carter L.
        • Page K
        Factors contributing to adverse events after ICU discharge: a survey of liaison nurses.
        Aust Crit Care. 2013; 26: 76-80
        • Gajic O.
        • Malinchoc M.
        • Comfere T.B.
        • et al.
        The stability and workload index for transfer score predicts unplanned intensive care unit patient readmission: initial development and validation.
        Crit Care Med. 2008; 36: 676-682
        • Iapichino G.
        • Morabito A.
        • Mistraletti G.
        • Ferla L.
        • Radrizzani D.
        • Reis Miranda D
        Determinants of post-intensive care mortality in high-level treated critically ill patients.
        Intensiv Care Med. 2003; 29: 1751-1756
        • Moreno R.
        • Miranda D.R.
        • Matos R.
        • Fevereiro T
        Mortality after discharge from intensive care: the impact of organ system failure and nursing workload use at discharge.
        Intensiv Care Med. 2001; 27: 999-1004
        • Smith L.
        • Orts C.M.
        • O'Neil I.
        • Batchelor A.M.
        • Gascoigne A.D.
        • Baudouin S.V
        TISS and mortality after discharge from intensive care.
        Intensiv Care Med. 1999; 25: 1061-1065
        • Hosein F.S.
        • Bobrovitz N.
        • Berthelot S.
        • Zygun D.
        • Ghali W.A.
        • Stelfox H.T
        A systematic review of tools for predicting severe adverse events following patient discharge from intensive care units.
        Crit Care. 2013; 17: R102
        • Kelley K.
        • Preacher K.J
        On effect size.
        Psychol Methods. 2012; 17: 137
        • Cohen J.
        Statistical Power Analysis for the Behavioral Sciences. 1988;
      1. Soper D. A-priori sample size calculator for multiple regression. Accessed 5/5/2019.

        • Knaus W.A.
        • Draper E.A.
        • Wagner D.P.
        • Zimmerman J.E
        APACHE II: a severity of disease classification system.
        Crit Care Med. 1985; 13: 818-829
        • Charlson M.
        • Szatrowski T.P.
        • Peterson J.
        • Gold J
        Validation of a combined comorbidity index.
        J Clin Epidemiol. 1994; 47: 1245-1251
        • Bartolomei C.
        • Cei M.
        The allocation of patients in an intensity-of-care based internal medicine ward: the ADOIT Tri-Co (triage in the corridor) study.
        Ital J Med. 2013; 1: 31-39
        • Armstrong E.
        • de Waard M.C.
        • de Grooth H.J.
        • et al.
        Using nursing activities score to assess nursing workload on a medium care unit.
        Anesth Analg. 2015; 121: 1274-1280
        • Pretto M.
        • Spirig R.
        • Milisen K.
        • Degeest S.
        • Regazzoni P.
        • Hasemann W
        Effects of an interdisciplinary nurse-led delirium prevention and management program (DPMP) on nursing workload: a pilot study.
        Int J Nurs Stud. 2009; 46: 804-812
        • Keers R.N.
        • Williams S.D.
        • Cooke J.
        • Ashcroft D.M
        Understanding the causes of intravenous medication administration errors in hospitals: a qualitative critical incident study.
        BMJ Open. 2015; 5e005948
        • Seki Y.
        • Yamazaki Y.
        Effects of working conditions on intravenous medication errors in a Japanese hospital.
        J Nurs Manag. 2006; 14: 128-139
        • Smith G.B.
        • Prytherch D.R.
        • Meredith P.
        • Schmidt P.E.
        • Featherstone P.I
        The ability of the national early warning score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death.
        Resuscitation. 2013; 84: 465-470
        • Smith G.B.
        Have we found the perfect early warning score? A view of views.
        Resuscitation. 2013; 84: 707-708
        • Prytherch D.R.
        • Smith G.B.
        • Schmidt P.E.
        • Featherstone P.I
        ViEWS–towards a national early warning score for detecting adult inpatient deterioration.
        Resuscitation. 2010; 81: 932-937
        • Griffiths P.
        • Ball J.
        • Bloor K.
        • et al.
        Nurse staffing levels, missed vital signs and mortality in hospitals: retrospective longitudinal observational study.
        Health Services and Delivery Research. Queen's Printer and Controller of HMSO, Southampton UK2018 (2016.38)
        • Salkind N.J.
        Encyclopedia of Research Design. 2. Sage, LondonUK2010
        • Polit D.F.
        • Beck C.T.
        The content validity index: are you sure you know what's being reported? Critique and recommendations.
        Res Nurs Health. 2006; 29: 489-497
        • Paiva C.E.
        • Barroso E.M.
        • Carneseca E.C.
        • et al.
        A critical analysis of test-retest reliability in instrument validation studies of cancer patients under palliative care: a systematic review.
        BMC Med Res Methodol. 2014; 14: 8
        • Khamiees M.
        • Raju P.
        • DeGirolamo A.
        • Amoateng-Adjepong Y.
        • Manthous C.A
        Predictors of extubation outcome in patients who have successfully completed a spontaneous breathing trial.
        Chest. 2001; 120: 1262-1270
        • Sanson G.
        • Sartori M.
        • Dreas L.
        • Ciraolo R.
        • Fabiani A
        Predictors of extubation failure after open-chest cardiac surgery based on routinely collected data. The importance of a shared interprofessional clinical assessment.
        Eur J Cardiovasc Nurs. 2018; 17: 751-759
        • Pittman R.N.
        Oxygen transport in normal and pathological situations: defects and compensations.
        Regulation of Tissue Oxygenation. Morgan & Claypool Life Sciences, 2011
        • Perel A.
        Non-invasive monitoring of oxygen delivery in acutely ill patients: new frontiers.
        Ann Intensiv Care. 2015; 5: 24
      2. World Health Organization. Haemoglobin concentrations for the diagnosis of anaemia and assessment of severity. Vitamin and Mineral Nutrition Information System. 2011(WHO/NMH/NHD/MNM/11.1) Accessed 5/5/2019.

        • Villar J.
        • Blanco J.
        • del Campo R.
        • et al.
        Assessment of PaO2/FiO2 for stratification of patients with moderate and severe acute respiratory distress syndrome.
        BMJ Open. 2015; 5e006812
        • Scott S.
        • Antonaglia V.
        • Guiotto G.
        • Paladino F.
        • Schiraldi F
        Two-hour lactate clearance predicts negative outcome in patients with cardiorespiratory insufficiency.
        Crit Care Res Pract. 2010; 2010917053
        • Lee S.M.
        • An W.S.
        New clinical criteria for septic shock: serum lactate level as new emerging vital sign.
        J Thorac Dis. 2016; 8: 1388-1390
        • Thongprayoon C.
        • Cheungpasitporn W.
        • Kittanamongkolchai W.
        • Harrison A.M.
        • Kashani K
        Prognostic importance of low admission serum creatinine concentration for mortality in hospitalized patients.
        Am J Med. 2017; 130 (e541): 545-554
        • Geoghegan P.
        • Harrison A.M.
        • Thongprayoon C.
        • et al.
        Sodium correction practice and clinical outcomes in profound hyponatremia.
        Mayo Clin Proc. 2015; 90: 1348-1355
        • Hessels L.
        • Hoekstra M.
        • Mijzen L.J.
        • et al.
        The relationship between serum potassium, potassium variability and in-hospital mortality in critically ill patients and a before-after analysis on the impact of computer-assisted potassium control.
        Crit Care. 2015; 19: 4
        • Vollam S.
        • Dutton S.
        • Lamb S.
        • Petrinic T.
        • Young J.D.
        • Watkinson P
        Out-of-hours discharge from intensive care, in-hospital mortality and intensive care readmission rates: a systematic review and meta-analysis.
        Intensiv Care Med. 2018; 44: 1115-1129
        • Hanley J.A.
        • McNeil B.J.
        The meaning and use of the area under a receiver operating characteristic (ROC) curve.
        Radiology. 1982; 143: 29-36
        • Youden W.J.
        Index for rating diagnostic tests.
        Cancer. 1950; 3: 32-35
        • Steyerberg E.W.
        • Harrell Jr F.E.
        • Borsboom G.J.
        • Eijkemans M.
        • Vergouwe Y.
        • Habbema J.D.F
        Internal validation of predictive models: efficiency of some procedures for logistic regression analysis.
        J Clin Epidemiol. 2001; 54: 774-781
        • Skalská H.
        • Freylich V.
        Web-bootstrap estimate of area under ROC curve.
        Austrian J Stat. 2006; 35: 325-330
      3. Skalská HFV. Bootstrap AUC (area under curve) estimation utility for ROC curve. Accessed 20/5/2019.

        • Pearson A.
        • Pallas L.O.
        • Thomson D.
        • et al.
        Systematic review of evidence on the impact of nursing workload and staffing on establishing healthy work environments.
        Int J Evid Based Healthc. 2006; 4: 337-384
        • Liu X.
        • Zheng J.
        • Liu K.
        • et al.
        Hospital nursing organizational factors, nursing care left undone, and nurse burnout as predictors of patient safety: a structural equation modeling analysis.
        Int J Nurs Stud. 2018; 86: 82-89
        • Lucchini A.
        • Elli S.
        • Bambi S.
        • et al.
        Nursing activities score: differences in nursing workload in three intensive care units.
        Assist Inferm Ric. 2015; 34: 6-14
        • Tiruvoipati R.
        • Botha J.
        • Fletcher J.
        • et al.
        Intensive care discharge delay is associated with increased hospital length of stay: a multicentre prospective observational study.
        PLoS One. 2017; 12e0181827
        • Lin F.
        • Chaboyer W.
        • Wallis M.
        • Miller A
        Factors contributing to the process of intensive care patient discharge: an ethnographic study informed by activity theory.
        Int J Nurs Stud. 2013; 50: 1054-1066
        • Oerlemans A.J.
        • van Sluisveld N.
        • van Leeuwen E.S.
        • Wollersheim H.
        • Dekkers W.J.
        • Zegers M
        Ethical problems in intensive care unit admission and discharge decisions: a qualitative study among physicians and nurses in the Netherlands.
        BMC Med Ethics. 2015; 16: 9
        • Chaboyer W.
        • James H.
        • Kendall M
        Transitional care after the intensive care unit: current trends and future directions.
        Crit Care Nurse. 2005; 25 (20-12, 24-16 passim; quiz 29): 16-18
        • Green A.
        • Edmonds L.
        Bridging the gap between the intensive care unit and general wards—the ICU Liaison Nurse.
        Intensiv Crit Care Nurs. 2004; 20: 133-143
        • Danesh V.
        • Neff D.
        • Jones T.L.
        • et al.
        Can proactive rapid response team rounding improve surveillance and reduce unplanned escalations in care? A controlled before and after study.
        Int J Nurs Stud. 2019; 91: 128-133
        • Niven D.J.
        • Bastos J.F.
        • Stelfox H.T
        Critical care transition programs and the risk of readmission or death after discharge from an ICU: a systematic review and meta-analysis.
        Crit Care Med. 2014; 42: 179-187
        • Rosa R.G.
        • Maccari J.G.
        • Cremonese R.V.
        • Tonietto T.F.
        • Cremonese R.V.
        • Teixeira C
        The impact of critical care transition programs on outcomes after intensive care unit (ICU) discharge: can we get there from here?.
        J Thorac Dis. 2016; 8: 1374-1376
        • van Sluisveld N.
        • Bakhshi-Raiez F.
        • de Keizer N.
        • et al.
        Variation in rates of ICU readmissions and post-ICU in-hospital mortality and their association with ICU discharge practices.
        BMC Health Serv Res. 2017; 17: 281
        • van Sluisveld N.
        • Hesselink G.
        • van der Hoeven J.G.
        • Westert G.
        • Wollersheim H.
        • Zegers M
        Improving clinical handover between intensive care unit and general ward professionals at intensive care unit discharge.
        Intensiv Care Med. 2015; 41: 589-604
        • Shiloh A.L.
        • Lominadze G.
        • Gong M.N.
        • Savel R.H
        Early warning/track-and-trigger systems to detect deterioration and improve outcomes in hospitalized patients.
        Semin Respir Crit Care Med. 2016; 37: 88-95
        • Papanikolaou P.N.
        • Christidi G.D.
        • Ioannidis J.P
        Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies.
        CMAJ. 2006; 174: 635-641