Central Line-Associated Bloodstream Infections (CLABSIs) remain a persistent and costly
challenge in U.S. hospitals, contributing to increased mortality, prolonged hospital
stays, and elevated healthcare costs. In 2022 alone, Providence St. Joseph Health (PSJH)
recorded 275 CLABSIs across 430,000 central line days. Despite the implementation of
best-practice prevention bundles, these infections continue to occur, prompting the
exploration of machine learning (ML) as a tool to predict and mitigate CLABSI risk. While
prior studies have demonstrated the predictive potential of ML models-with area under the
curve (AUC) values reaching up to 0.87-no randomized trial has yet evaluated the
real-world clinical impact of deploying such a model.
The primary objective of this trial is to determine whether the deployment of a machine
learning model that predicts possible CLABSI risk, when provided to hospital Infection
Preventionists (IPs) with a standardized workflow, can reduce CLABSI rates compared to
routine practice. Secondary objectives include assessing the intervention's impact on
central line removal within 48 hours of an alert, the rate of positive blood cultures,
and various process metrics such as the frequency of IP interventions. Safety outcomes,
including pneumothorax and hemorrhage, are also being monitored.
The study is designed as a prospective, open-label, multi-center, cluster-randomized
controlled trial conducted across 20 Providence hospitals with the highest CLABSI burden.
These hospitals account for approximately 90% of all CLABSI events within the system
during 2023 and 2024. Hospitals were paired using Mahalanobis distance based on the
hospital's CLABSI count and NHSN Standardized Infection Ratio (SIR) and then randomized
into early and late intervention groups. The early group received access to the ML model
for four to five months before the late group. Infection Preventionists at early
hospitals used a dashboard to identify high-risk patients and deliver targeted education
and interventions focused on central line care.
The machine learning model was developed using data from over 62,000 patients and more
than 730,000 line-days collected between January 2015 and September 2024. A positive
class was defined as a positive blood culture occurring within 24 to 72 hours in a
patient with a central line in place for more than 48 hours. From 87 electronic medical
record (EMR) data elements, 207 features were engineered for model development. The
modeling process employed XGBoost and addressed class imbalance through oversampling,
undersampling, and SMOTE techniques. The final model achieved an AUC of 0.93, with a
recall of 0.72, precision of 0.66, and an F1 score of 0.68. To ensure fairness, the model
underwent a bias analysis using the EEOC's four-fifths rule, confirming consistent
performance across race, sex, and ethnicity subgroups.
Each day, the model scored all adult inpatients with central lines in place for more than
48 hours. Predictions were published to a PowerBI dashboard accessible to IPs at
intervention hospitals. These IPs reviewed flagged patients, ensured adherence to the
CLABSI prevention bundle, and recommended line removal when appropriate. The IPs actions
were documented in the EMR. The intervention was supported by training, scripting for
clinical conversations, and access to infectious disease physicians for consultation.
The primary outcome of the trial is the CLABSI rate, defined as events per 1,000 central
line-days and adjudicated using NHSN criteria. Secondary outcomes include the proportion
of lines removed within 48 hours of a model alert, the rate of positive blood cultures,
the rate of possible CLABSIs (defined as a positive culture in a patient with a line in
place for more than 48 hours), and the total number of central line days per hospital.
Additional metrics include the frequency of IP interventions and before-after comparisons
of CLABSI rates.
The statistical analysis plan centers on a generalized linear mixed model (GLMM), using
either a Poisson or Negative Binomial distribution depending on the presence of
overdispersion. The model includes a log of line-days as an offset and incorporates
hospital as a random effect to account for clustering. Fixed effects include group
assignment and calendar month. Covariates are included to improve precision and control
for confounding. Hospital-level covariates include hospital type, medical school
affiliation, average length of stay, total bed count, and ICU bed proportion.
Patient-level covariates include age, race and ethnicity, primary payer, history of
CLABSI, line type, and line location. Sensitivity analyses will explore the additional
comorbidities such as immunosuppression, obesity, diabetes, and diarrhea. Inverse
Probability of Treatment Weighting (IPTW) will be considered to further adjust for
confounding.
Sample size calculations were based on a baseline CLABSI rate of 0.004 events per patient
per month, with an intra-cluster correlation (ICC) of 0.05 and a targeted 20% relative
risk reduction (RR = 0.8). Under these assumptions, approximately 8,920 patients per arm
are required to achieve 80% power at a significance level of 0.01. The study duration was
set at five months to accrue the necessary 64 CLABSI events. An interim analysis is
planned at 2.5 months, using the O'Brien-Fleming group-sequential design to allow for
early stopping due to efficacy or harm. The interim analysis will apply a nominal p-value
threshold of 0.0088, while the final analysis will use a threshold of 0.0467 to maintain
an overall Type I error rate of 5%.
