Madsen K, Rosenman M, Hui S, Breitfeld PP. Value of electronic data for model validation and refinement: bacteremia risk
in children with fever and neutropenia. J Pediatr Hematol Oncol 2002;24(4):256-62.
BACKGROUND: Validating published risk models in a different time and setting can be a labor-intensive process. Data in
electronic format provide the potential to test the validity of risk models without labor-intensive chart reviews and
data capture. The authors attempted to use readily available electronic data to find appropriate cases and to validate
and refine a previously developed risk model for predicting bacteremia in children with cancer who had fever and neutropenia.
PATIENTS AND METHODS: By applying a largely automated case-finding algorithm to linked, electronic clinical and administrative
data systems, the authors identified and acquired data regarding 157 episodes of fever and neutropenia in children with cancer
admitted to a children's hospital during an 11-month period in 1997. The authors applied a previously developed and validated
risk model for bacteremia to this 1997 cohort by assessing the odds ratios among risk groups. The model assigns encounters with
absolute monocyte count of 100 cells or more/mm3 to a low-risk group and encounters with an absolute monocyte count of less than
100 cells/mm3 to intermediate-risk (temperature <39.0 degrees C) or high-risk (>/=39.0 degrees C) groups. In addition, the authors
explored whether the new data would have generated the same model. Univariate and multivariable analyses were performed to
determine whether there were additional independent predictors of bacteremia. Recursive partitioning of admission absolute
monocyte count and temperature was used to assess whether similar cutpoints would be found.
RESULTS: There were 12 episodes of bacteremia (7.6%) among the 157 encounters. The previously developed model correctly
predicted increasing rates of bacteremia in this 1997 cohort, ranging from 2.5% in the low-risk group (one episode in a
child with an infected central line) to 24% in the high-risk group. The odds ratio for the high-risk versus intermediate-risk
group was 4.09 (95% confidence interval 1.05-15.91), comparable to the odds ratio of 3.96 in the previously published derivation
cohort (95% confidence interval 1.4-11.1). Multivariate analysis of the new data revealed no independent risk factors for
bacteremia other than admission absolute monocyte count and temperature. Recursive partitioning of absolute monocyte count
and temperature generated risk categories that were somewhat different from those of the original model. The new data yielded
three categories: low risk (temperature 10/mm3), intermediate risk (temperature
39.5 degrees C).
CONCLUSIONS: Existing electronic data provide an efficient means for case-finding and model validation and refinement. The
previously developed bacteremia model had good but not optimal predictive performance in the new data set. Admission absolute
monocyte count and temperature remain significant risk factors for bacteremia. Redefining the risk categories, including a
much lower cutpoint for admission absolute monocyte count, improved the model's discrimination, which suggests that predictive
models need periodic updating.
Email us comments or suggestions