Study design and setting

We utilise data from patients admitted to paediatric wards across 13 county (formerly district-level) hospitals in Kenya. A detailed description of the selection of hospitals in this study has been reported elsewhere [1]. In brief, these hospitals were purposefully selected to represent high and low or very low malaria prevalence settings and spanning large urban environments and rural county towns that are likely to admit at least 1000 children per year. Collectively, hospitals in this study are part of the Clinical Information Network (CIN) which is designed as a collaboration between researchers from the KEMRI-Wellcome Trust Research Programme, the Kenya Ministry of Health, the Kenya Paediatric Association and the University of Nairobi. One smaller hospital of the 14 in CIN not designated as a county referral hospital and with no attending physicians was excluded from these analyses as this site has a different complement of nurses and different patient case-mix. A principal aim of the CIN is to use de-identified patient-level data to begin to understand and improve hospital care. The work undertaken by the CIN has received approval from the national Kenya Medical Research Institute scientific and ethical review committees.

Study population, data collection and management

All patients aged ≥1 month hospitalized in the paediatric wards of 13 hospitals were eligible for inclusion from September 2013 through April 2016. Patients with surgical conditions or burns were excluded ( Figure 1 ). Data were abstracted on discharge or death by a trained data clerk from each of the patient records, aided by clinicians’ use of a Paediatric Admission Record (PAR) form [22], into a customised data capture tool designed in the non-proprietary Research Electronic Data Capture (REDCap, Vanderbilt University, USA; https://www.project-redcap.org/) platform [23]. Data captured and used in these analyses included: basic biodata, history, examination findings, initial vital signs measures recorded in the admitting clinician’s record, diagnosis (defined by the International Classification of Diseases-Tenth Revision (ICD-10) codes), and measurements recorded by nurses on the patient vital signs monitoring chart. The latter data focused on all temperature (T), pulse (P) and respiratory rate (R) measures recorded in the first 48 hours of admission, the time of highest mortality [1]. Data from all 13 hospitals were synchronized to a central server on a daily basis. A full description of the data collection procedures, data quality monitoring, validation framework and the web-based data synchronization is provided elsewhere [24].

Figure 1.  Populations used in different analyses.

Analysis

We first explored whether an “admission set” of vital signs (T, P, R) were recorded in the medical record by the clinician on duty at admission. The primary outcome for other analyses was the count of subsequent vital signs taken by nurses determined by counting the number of times temperature (T), pulse (P) and respiratory rate (R) were documented on the nursing vital signs chart during the initial 48-hour period. The nurses’ vital signs count was then used in two separate analyses: (a) comparison of the median patient-level count for each hospital with a standard representing the minimum count expected during this initial period of inpatient stay; (b) negative binomial regression analysis to determine the relationship between vital sign (count) and signs of severe illness, admission syndromic diagnosis, child’s age and outcome of admission at patient level. The standard for comparisons of vital signs counts was derived from consensus discussions with senior nurses from the CIN hospitals (collaborators). It was agreed that a reasonable standard, in addition to the ‘admission set’ of clinical signs documented in the medical record, was to have a minimum of 9 observations (3T, 3P and 3R) in 24 hours (representing one set of vital signs observations per nursing shift) and consequently a minimum of 18 observations in 48 hours. As times of admission, death or discharge are not routinely available we stratified patients into different groups. These include those that died on the admission date (within 24 hours) and one or two days after the admission date (Day 1 deaths, approximately 24 hours stay; Day 2 deaths, approximately 48 hours stay). Children who survived the initial 48-hour of admission were further stratified into groups of those with and without danger signs on admission as defined by WHO and Kenyan guidance [25,26].

Data are described using medians, interquartile ranges (IQR), and proportions where appropriate. We use graphical presentations of vital signs values and counts to examine the variability across patient groups and hospitals and Venn diagrams to examine the pattern of admission vital signs recording by clinicians. We used a right-tailed binomial test to test a null hypothesis of no end-digit preference when clinicians record vital signs observations at admission (for example, overall we would expect the number of respiratory rate observations that are even numbers to equal those that are odd numbers, on average).

To explore factors that influence variability of the vital signs count across hospitals, we restricted analyses to patients who had an inpatient stay of at least 48 hours with survival status documented and who had a nursing vital sign chart present in the medical record (n = 41 738, Figure 1 ). We hypothesised a priori patient-related factors that might affect frequency of vital signs monitoring in paediatric wards. These factors were based on the clinical judgement of the authors and included: i) patient age group, ii) subsequent survival status (alive/dead, in this case we consider that this later outcome may be a marker for clinicians/nurses ability to recognise factors that provide them with an initial ‘gut feeling’ about severity of patients’ illness [27] that may influence their actions), iii) indicator variables (taking the value 0 or 1) for each admission diagnosis with a ‘severe disease classification’ spanning malaria, pneumonia, anaemia, dehydration, malnutrition, or meningitis (Kenyan and WHO guidelines provide criteria for assigning level of severity [25]) and a distinct indicator variable for those cases with multiple severe illnesses (also coded 0/1), and iv) an indicator variable (taking the value 0 or 1) for each clinical danger sign from those highlighted in Kenyan guidelines including altered level of consciousness (value<A on Alert Voice Pain Unresponsive (AVPU) scale), history of convulsion, or presence of central cyanosis, severe pallor, vomiting everything, acidotic breathing or inability to drink/breastfeed and a distinct indicator variable for those cases with multiple danger signs (also coded 0 / 1). Child’s sex was also included in the adjusted regression model. Patients who died or that were discharged within 48 hours were excluded from analyses of associations with monitoring frequency because we did not have data on their length of stay in hours. The number of vital signs observations would be related to the actual length of stay for those not completing a full 48-hour stay in hospital, potentially confounding analyses of associations of patient characteristics with counts of monitoring episodes over 48 hours.

Handling missing data

Before fitting regression models we explored the levels of missingness both in explanatory and dependent variables. While level of missingness in some of the explanatory variables was not of concern (<1%) for the primary outcome of vital signs count 20.3% cases had data on any one of the T, P or R measurements missing / not recorded. We therefore used multiple imputation to address missingness for individual T, P and R counts using chained equations [28,29] under the assumption of missing at random (MAR) before computing the vital signs count for each case. The imputation models included both key variables (as used in the analysis model) and auxiliary variables (other relevant variables included to add extra information to the imputation model as is recommended [29]). Key variables included: gender, age group, hospital identity, severe childhood illness, danger signs, and survival status of a patient. Auxiliary variables included: number of nurses per shift, ward bed capacity, temperature readings at admission, and season. Since the number of vital signs measurements in 48 hours at the point of data entry could take discrete values ranging from 0 to 10 for each of T, P and R, an ordered logit model [29] was deemed appropriate and hence used in the imputation. The simulation error was minimised by using 50 imputations with 100 iterations. We further assessed our imputation approach using convergence plots and plots of the marginal distribution of both imputed and observed values as is advised [29] and conducted sensitivity analyses to investigate the validity of the MAR assumption using pattern mixture models [30]. These procedures and results of these analyses are presented in Online Supplementary Document(Online Supplementary Document) . We concluded that the imputation was satisfactory and our assumption of MAR reasonable.

Model specification

We used mixed effects models to account for clustering with hospitals included as random effects in all regression models. As vital signs counts take discrete, nonnegative values and because the Poisson model assumption of equidispersion was violated, analyses employed negative binomial regression models that account for over-dispersion. Analyses were first conducted for each of the explanatory variables to examine associations with vital signs count. All variables were then included in multivariable mixed effects regression models with clinically relevant interactions explored using likelihood ratio tests. No interactions were found to be significant and results indicated that multi-collinearity was not a concern. Both univariate and multivariable models were fitted using 50 multiply imputed data sets. Visual inspection of residual plots did not show obvious departures from the model assumptions. We therefore derived final estimates of the mixed effects negative binomial regression models (univariate and multivariable) pooled from all multiply imputed data sets using Rubin’s rules [31]. We conducted a sensitivity analysis to explore the consistency of our estimates as follows; we fitted our final model using a data set (n = 42 500) where patients with missing data on their outcome (alive/dead) had been multiply imputed using chain equations. The estimates for associations from these sensitivity analyses were not appreciably different from those we report using the data set (n = 41 738) that excluded patients without outcome (alive/dead) data. All analyses were performed using R Version 3.2.5 (R Foundation for Statistical Computing, Vienna, Austria; http://www.cran.r-project.org).

Population characteristics

The eligible study population consisted of 54 800 patients across 13 CIN hospitals from the period September 2013 through April 2016 and is described in Table 1 . The median age of the study population was 20 months (IQR 10-48) and 54.6% were male. Overall inpatient mortality was approximately 6.1% but proportions varied considerably across hospitals (range 2.3-10.3%). The median number of beds per pediatric ward was 35 (IQR 32-41) that were attended by a median number of 2 (range 1-2) fully qualified nurses per shift. Hospital H5 with mortality approximately 3% had the highest median vital signs count of 30 while hospital H10 with a mortality of approximately 10.3% had the lowest median count of 2 for those with hospital stays of ≥48 hours ( Table 1 ).

Vital signs at admission

Clinicians recorded a full set of vital signs (TPR) for 57% of children on admission, 74% of whom were reported to have fever, while in 8.4% none of the vital signs were documented. It was more common to have a single temperature observation (10.4% of admissions) or a combination of temperature and respiratory rate observations (16.8% of admissions) where a full set of vital signs was not recorded ( Figure 2 ). Although measurement of blood pressure is also regarded as a key vital sign in sick children we abandoned collecting these data as it was recorded in fewer than 2% admissions.

Figure 2.  Proportion of children whose vital signs were documented at admission are represented by the oval shapes. Intersections of the ovals represent proportions of children who had either 2 or all 3 vital signs documented at admission while sections of the ovals without intersection represent proportions of children who had only 1 of the three vital signs documented. Proportion of children who had none of three vital signs documented are presented as “None”.

Vital Signs Count variability

Variability in the vital signs count across different hospitals for different populations is illustrated in Figure 3 . As expected counts were lower in those dying on the day of admission than those who were discharged alive on the day of admission. This trend was consistent for the patients who died and who survived for 24 hours. For patients who died two days after admission the median of all 13 hospitals’ vital signs counts was still below the recommended 24-hour standard of 9. Amongst those surviving at least 48 hours the median of all 13 hospitals’ vital signs counts for patients with danger signs at admission was 9 (range of hospital medians 0-30) compared with a recommended 18 and this estimate did not differ for populations with and without danger signs. Some hospitals (H5, H7 and H9) achieved consistently high vital signs counts. Thus, surpassing the recommended 18 in 48 hours while others were consistently low (eg, H3 and H10). Exploring the counts for individual vital signs ( Figure 4 ) suggests that pulse and respiratory rate measurements were more likely to be left undone across time with almost half of children having had either none, one or two pulse and respiratory rate measurements over the first 48 hours (compared with an expected count of 6). Further between 15 and 25% children who died on the admission day or the day after admission had no pulse or respiratory rate observations recorded ( Figure 4 ).

Figure 3.  Variability of vital signs count in different populations in the first 48 hours of admission across hospitals under study. Each dot is the median vital sign count for individual hospitals in a specific population and the diamond is the median of these medians. Asterisk (*) represents hospitals of high mortality. “DS” denotes danger signs.

Figure 4.  Distribution of proportions of the number of times each vital sign (T, R, P) was monitored by nurse(s) in different populations(alive/dead) across time within 48-hour period

Validity of vital signs observations: digit preference

Plots for vital signs values recorded at admission for each hospital showed a similar pattern. As a result, we present data pooled across hospitals ( Figure 5 ). Visual inspection of the distribution of individual vital signs readings for respiratory rate and pulse rate showed marked peaks. For instance, reported readings of 40 and 120 for respiratory rate and pulse rate, respectively, were very common. There was no evidence for even/odd end-digit preference in temperature readings although there were clear peaks at values of 36.7°C, 37.0°C, 38.0°C, 39.0°C and 40.0°C.

Figure 5.  Distribution of individual vital signs readings at admission pooled across all hospitals.

Respiratory and pulse rate readings showed a high prevalence of even end-digits, approximately 79% and 77% respectively (see Online Supplementary Document(Online Supplementary Document) ) that is highly unlikely to have arisen by chance (right-tailed binomial exact test in both cases P < 0.001).

Multivariable analysis

In the population of patients included in multivariable analysis ( Figure 1 , n = 41 738) higher vital signs counts were weakly associated with the presence of multiple severe illnesses in a patient (adjusted risk ratio (aRR) of 1.04 (95% CI = 1.02– 1.06), P < 0.01), but not with any individual severe illness diagnosis. Amongst the danger signs that we considered, severe pallor was the only sign that was significantly associated with an increase of 5% in the vital signs count (aRR = 1.05 (95% CI = 1.01– 1.09), P = 0.02). There was no evidence to suggest a relationship between patients’ age, gender or outcome after Day 2 of admission with vital signs count ( Table 2 ). The association between frequency of monitoring of each vital sign (T, P, R) considered as separate dependent variables and potential explanatory covariables was consistent with the pattern observed for the summary vital signs count (Table S1, S2 and S3 in Online Supplementary Document(Online Supplementary Document) ).

Limitations and strengths of the study

Our study has a number of limitations. Data were collected after patient discharge from medical records. We therefore capture only what is documented. However, we have developed rigorous procedures for such record review over a period of years [46] and employ timely data quality checking procedures [24]. Nurses may not document observations actually collected on the vital signs chart, and we encountered moderate levels of missingness when trying to construct a vital signs index. However, senior nurses agreed that the vital signs chart is the primary clinical record for observations as this is the tool used to share information across nursing and medical teams. To account for missingness we used multiple imputation to ensure maximum use of available data examining the robustness of this approach. The hospitals included in the CIN are also not a statistically representative sample of such facilities in Kenya, they have varying case mix [1] and over time have received regular feedback on their performance (although there has been little attention paid to nursing observations to date) [24]. Our results may therefore not be generalizable to other settings. They are also centres providing experiential training for junior or student clinicians and nurses. The latter often augment staffing on wards in support of qualified nurses. This may mean that the results we report are better than might be found in many non-training centre hospitals in Kenya or possibly worse if students record vital signs inaccurately.