In many studies, continuous or categorical predictors are collapsed into a binary variable by using threshold values. For example, the association between age and outcome has frequently been analyzed at a threshold value of 50 years. This approach has disadvantages.⁷ First, it is unnatural: would risks really be very different for patients who had their 50th birthday yesterday versus having their birthday tomorrow? In addition, a patient 30 years of age may have a different risk than a patient 49 years old. Yet both are below a threshold of 50. Second, from a methodologic perspective, collapsing an ordinal or continuous scale into a binary variable (dichotomization) leads to loss of information and is therefore statistically inefficient. In general, it is better to exploit the full information available. The same principle applies to the approach to analysis of outcome.

Missing Data

Missing data are a common, but as yet underappreciated problem in medical scientific research. Missing values lead to a more limited set of patients with complete data as opposed to the ideal situation of complete original data.

The best solution for missing values is of course to ensure that no data are missing. If we nonetheless have missing data, a common statistical approach is to delete patients with missing values from the analysis. This is often referred to as a complete case analysis.^8,⁹ Complete case analysis discards information from patients who have information on some (but not all) predictors. It is hence statistically inefficient, especially when we consider multiple predictors. Moreover, complete case analysis may lead to bias because of systematic differences between patients with complete and patients with missing data. Bias occurs when absence of a predictor is associated in some way with the outcome.¹⁰

Most statisticians now agree that we may opt for a more sophisticated statistical approach to deal with missing values in predictive regression models: single or multiple imputation.¹¹ Imputation methods substitute the missing values with plausible values so that the completed data can then be analyzed with standard statistical techniques. As in any statistical analysis, sensible judgement of the analyst based on knowledge of the subject and the research question is important. In practice, many clinicians are unaware of the problems inherent in complete case analyses and are ignorant of modern developments for dealing with missing data, in particular, the use of multiple imputation methods.

Outcome Measures for Prognostic Studies in Patients with Traumatic Brain Injury

In prognostic analysis, the outcome measure chosen should be clinically relevant, and “hard” end points are generally preferred. Mortality is often used as an end point in prognostic research, but global outcome measures (e.g., GOS), nonfatal events (e.g., disease recurrence), patient-centered outcomes (e.g., scores on quality-of-life questionnaires), or wider indicators of burden of disease (e.g., absence from work) may also be used. Whatever the end points chosen, assessment at a fixed time point is essential. Statistical power may also direct the choice of outcome. When an outcome is very infrequent, it is not suited as an end point for statistical analysis.

Most, if not all prognostic studies of TBI have used the GOS or mortality as end points. In most cases the GOS, which may be considered an ordinal scale with five categories, was collapsed into a dichotomous variable differentiating unfavorable from favorable outcome (Table 340-1). In the use of a dichotomous outcome measure, statistical power is greatest when there is a 50 : 50 distribution between outcome categories.

TABLE 340-1 Glasgow Outcome Scale

In TBI it is better to quantify prognostic effects across the full range of the GOS rather than after dichotomization into a binary variable. For this purpose, proportional odds methodology is appropriate.¹² The 8-point extended GOS (GOSE) has been introduced to increase the sensitivity of outcome assessment. The use of a structured interview is further advocated to obtain more consistency in outcome assignment.¹³ In severe TBI, the outcome distribution according to the 5-point GOS is U shaped, with most patients either in the lowest (dead) or highest (good recovery) categories. This U-shaped distribution of outcome has promoted the common practice of dichotomizing the GOS for analysis. There is still insufficient knowledge on how introduction of the GOSE may have changed the outcome distribution. It should be noted that the potentially increased sensitivity of the GOSE is totally lost when this is again dichotomized to a binary scale.

Despite the increased sensitivity of the GOSE, it remains a fairly global scale with broad categories aiming to capture functional reintegration without discriminating between physical and mental disabilities. More specific tools for outcome assessment include the Functional Independence Measure (FIM),¹⁴ neuropsychological tests, and quality-of-life assessments. The FIM is considered more appropriate for monitoring the course of a patient through rehabilitation, and neuropsychological tests are considered mandatory for assessing cognitive function, which is so often disturbed after TBI. Generic health-related quality-of-life measures, such as the 36-item short-form health survey (SF-36), are routinely used in many fields of medicine but have not yet become common in assessment of outcome after TBI. These generic health scales may not actually capture relevant domains after TBI, and for this reason the disease specific QOLIBRI scale (Quality of Life after Brain Injury) has been introduced.¹⁵ By definition, quality-of-life scales report the subjective experience of the patient or caretaker and consequently provide a different perspective from the possibly more objective assessment by health care professionals. Currently, there is much interest in the development of composite outcome measures for TBI in which the advantages of more detailed information from the components can be summarized. Further research is required to determine the ultimate benefits from this approach, both for prognostic analysis and in the context of clinical trial design.

Approaches to Prognostic Analysis

The first step in prognostic analysis is identification of the association between a single prognostic factor and outcome (univariate analysis). We might ask, for example, what is the effect of motor score at admission on the 6-month GOS? It should be stressed that a univariate association does not take into account the role of other predictors that may be more important or even may account for the observed association. This association does not represent causality, and the association may be secondary to other more relevant predictors.

The second step, therefore (multivariate analysis), focuses on the unique predictive value of that predictor over and above that of other covariates. Questions that require multivariable analysis are, for example, what are the most important predictors in a certain disease? Are some predictors correlated with each other such that their apparent predictive effects are explained by other predictor variables? To perform multivariate analysis, more predictors are added to the regression model as independent variables.

The third step (prognostic modeling) depends on combining information from the different individual prognostic features into a prognostic model with the aim of giving the best predictions for individual patients. The relevance of a predictor is a function of association of the predictor with the outcome and the distribution of the predictor. For example, a dichotomous predictor with an odds ratio (OR) of 2.0 and a 50% prevalence is more relevant for a prediction model than a dichotomous predictor with an OR of 2.5 and a 1% prevalence.

Predictors for inclusion in the model are usually selected from a stepwise selection procedure: we define a P value to include or exclude predictors, and the statistical package defines the final regression model based on this P value.

The relationship between predictors and outcome can be quantified in several ways (Tables 340-2 and 340-3).

TABLE 340-2 Performance Measures of Predictors

MEASURE	DEFINITION	INTERPRETATION
Relative risk (RR)	Risk for outcome in a group with a predictor/Risk for outcome without a predictor	For example, an RR of 2 means that the group with the predictor has twice the risk of the group without the predictor. When the predictor is continuous, RR represents the increase per unit
Odds ratio (OR)	Ratio of the odds for a better versus a poorer outcome in the presence of the parameter (a/b) as compared with the odds in the absence of the parameter (c/d)	If the prognostic factor is not associated with outcome, the OR will be 1. In reporting the OR, the 95% confidence interval (CI) is frequently included. Statistical significance of the relationship is present if the CI does not include the value 1
R²	Model sum of squares/Total sum of squares (Sum of squares is parameters of the regression model)	Percentage of variability in the outcome that is explained by the predictor or predictors
Sensitivity	Number of true positives/Total number with the outcome	Proportion of patients with the outcome who have the predictor (true positive)
Specificity	Number of true negatives/Total number without the outcome	Proportion of patients without the outcome who do not have the predictor (true negative)
Positive predictive value (PPV)	Number of true positives/Number of positives	Proportion of patients with the predictor who do have the outcome
Negative predictive value (NPV)	Number of true negatives/Number of negatives	Proportion of patients without the predictor who do not have the outcome.

Data from Vittinghoff E. Regression Methods in Biostatistics: Linear, Logistic, Survival, and Repeated Measures Models. New York: Springer; 2005; and Altman DG. Practical Statistics for Medical Research. New York: Chapman & Hall; 1991.

TABLE 340-3 2 × 2 Table for Explanation of Performance Measures

	DEAD	ALIVE
Predictor present	a	b
Predictor absent	c	d

In the first edition of the Guidelines on Management and Prognosis of Severe Head Injury published by the Brain Trauma Foundation in 2000, the positive predictive value (PPV) was used as main measure for expressing prognostic performance. The PPV, however, has limited value because it does not take the frequency at which a predictor occurs within the population into account. Currently, the most widely used measure for expressing the strength of association in prognostic analysis is the OR, which can be obtained directly from the output of a regression model. In multivariable analysis, the ORs provided by the regression model are adjusted for the other predictors in the model. The R² is also provided by the output of the regression model. The difference in R² between a model without and with a certain predictor is the percentage of the variance that is explained by that predictor above the predictors in the former regression model.

Building Blocks for Prognostic Analysis

A wealth of literature has focused on the associations between predictors and outcome in univariate analysis. Most studies have concentrated on patients with severe and moderate TBI. Fewer studies have included multivariable analysis, and two systematic reviews on prognostic modeling have shown the shortcomings of many of the studies that reported on prognostic models previously. Much information on the univariate association between predictors and outcome is contained in the section “Early Indicators of Prognosis in Severe Traumatic Brain Injury” of the Brain Trauma Foundation’s Guidelines on Management and Prognosis of Severe Head Injury, first published in July 2000. More recently, the IMPACT study group reported the results of extensive prognostic analyses performed in a meta-analysis of individual patient data from eight randomized controlled trials and three observational series that included more than 9000 patients.¹⁶ A series of papers reported details correlating the GOS and demographic characteristics,¹⁷ cause of injury,¹⁸ GCS and pupil response,¹⁹ secondary insults,²⁰ blood pressure,²¹ computed tomography (CT) scan features,²² and laboratory parameters.²³ The results of multivariable analyses reporting also on the added predictive value were presented in the same series by Murray and colleagues.²⁴

Conceptually, the main predictors of outcome after TBI can be grouped together into “building blocks,” some of which are modifiable and some not (Table 340-4).

TABLE 340-4 Building Blocks for Prognostic Analysis

BUILDING BLOCKS	ITEMS	MODIFIABLE
Genetic constitution	Apolipoprotein E	No
Demographics	Age Gender Race	No
Clinical severity	Glasgow Coma Scale score Pupillary reactivity Extracranial injuries	No
Secondary insults	Hypotension (blood pressure) Hypoxia Hypothermia	Yes
Structural abnormalities	CT classification Traumatic subarachnoid hemorrhage Type of intracranial lesion	Sometimes
Laboratory parameters	Glucose Sodium pH Coagulation parameters Hemoglobin	Yes
Biomarkers	Items under development	Uncertain

CT, computed tomography.

The strength of the association of these predictors with outcome in both univariate and multivariate analyses as reported by the IMPACT investigators is summarized in Table 340-5.

TABLE 340-5 Strength of the Association between Predictors and Outcome after Traumatic Brain Injury

Current knowledge on these “building blocks” and parameters is summarized in the following sections.

Genetic Constitution

In this era of discovery of the human genome, several genes and their polymorphisms are under investigation in patients with TBI. The presence of the apolipoprotein E4 allele is associated with poorer functional recovery.²⁵ Other genes for which evidence exists for an association with poorer outcome are P53, COMT, DND2, and CACNA1A.²⁶

Demographic Factors

Age is the strongest and most extensively studied predictor of outcome after TBI. Many publications on prognostic effects exist, all stating that older age is correlated with poorer outcome (Fig. 340-1). It is remarkable that most studies have analyzed the association between age and outcome with threshold values. Different thresholds varying from 30 to 60 years of age have been used.²⁷^–³⁷ Studies using higher threshold levels report poorer outcome in the upper age group, and a mortality rate of greater than 75% has been described in patients older than 60 years with severe TBI.³⁸^–⁴⁰ A continuous age dependency was described in only few publications in the past.⁴¹^–⁴³

FIGURE 340-1 Continuous association between age and outcome.

(From Mushkudiani NA, Engel DC, Steyerberg EW, et al. Prognostic value of demographic characteristics in traumatic brain injury: Results from the IMPACT study. J Neurotrauma. 2007;24:259-269.)

In a recent meta-analytic study of individual data from more than 9000 patients, a continuous effect of age on outcome was described that could be approximated by a linear function. Threshold values could not be identified. Other demographic factors studied for their association with outcome include gender, race, and education.

Males are more prone to sustain a TBI because of higher risk for road traffic accidents and assaults, but a clear association between gender and outcome assessed by the GOS has not been shown for TBI. This, however, does not exclude the possibility that there may be some effect of gender on outcome. In a meta-analysis, Farace and Alves found poorer outcome in females surviving severe TBI than in males.⁴⁴ Experimental studies, however, indicate that progesterone may have a neuroprotective effect, and consequently, prognosis in females may be better if TBI is sustained at a time of the menstrual cycle when progesterone levels are high.

The possible association between race and outcome after TBI has not been extensively studied. Two smaller studies showed poorer outcome in black patients,^45,⁴⁶ but others did not find a clear association.^34,^47,⁴⁸ The IMPACT study group, in contrast, studying data from 5320 patients, found a statistically significant association between race and outcome, with black patients having a poorer outcome.¹⁷ It was hypothesized that this might be due to differences in severity or cause of injury. Such, however, proved not to be the case, and after adjustment for cause of injury, age, motor score, and pupils, the prognostic effect was even stronger (Table 340-6).

TABLE 340-6 Strength of the Association between Race and Outcome after Traumatic Brain Injury

TYPE OF ANALYSIS	ODDS RATIO	CONFIDENCE INTERVAL
Univariate	1.30	1.09-1.56
Adjusted for cause of injury	1.31	1.08-1.57
Adjusted for age, motor score, and pupils	1.44	1.08-1.93
Adjusted for 7 clinical predictors*	1.45	1.07-1.96

* Age, Glasgow Coma Scale motor score, pupils, hypoxia, hypotension, computed tomographic classification, traumatic subarachnoid hemorrhage.

These data are convincing and leave little room for doubt, but what the underlying reason for this association is can only be speculated. The response to injury may be different because of genetic constitution and biologic differences, or possibly access to acute ⁴⁹ and postacute⁴⁷ care may be more limited.

A weak association between level of education and outcome has been reported.^17,⁴⁶ Mushkudiani and coworkers, however, clearly showed that this weak association could be explained by other factors and disappeared on adjustment.

Clinical Severity

Clinical severity is an important prognostic factor that in theory at least, can be assessed in all patients. Severity in patients with TBI is related to both extracranial and intracranial injuries. The overall severity of extracranial injuries is commonly assessed with the Abbreviated Injury Score (AIS)⁵⁰ or the Injury Severity Score (ISS).⁵¹ Although an adverse effect of the presence of extracranial injuries on outcome after TBI is well documented,⁵² the prognostic value of overall injury severity and extracranial injuries has not been well documented in the literature. Walder and associates found a stronger association between the AIS and outcome than assessment of the GCS alone.⁵³

The clinical severity of intracranial injuries is reflected by the level of consciousness as assessed by the GCS.¹ Many studies have demonstrated an association between lower scores on the GCS and poorer outcome. In patients with more severe injuries, the motor component of the GCS has the greatest predictive value because eye and verbal responses are commonly absent in these patients. It should be recognized that GCS scores may fluctuate early after injury, with some patients deteriorating and others improving. From a perspective of prognosis, assessment of the GCS should therefore be related to a given time period, depending on the intent for estimating prognosis. For purposes of classification and summarizing baseline characteristics before in-hospital therapeutic interventions, the GCS is commonly assessed on admission after primary respiratory and hemodynamic stabilization. Reliable assessment of the GCS, however, may be obscured in the acute setting by confounders such as medical sedation, paralysis, or intoxication.^19,^54,⁵⁵ Marmarou and coauthors reported a stronger association with outcome for an abnormal absent motor response than for an absent motor response.¹⁹ The most likely explanation for this is that the category of patients scored as having an absent motor reaction will include “false-absent” scores because of the confounding effects of sedation and paralysis.

Abnormalities in pupillary reactivity reflect brainstem compression and are strongly associated with poorer outcome.¹⁶ Marmarou and colleagues reported that pupillary reactivity was a more stable parameter in the early phase after injury than the GCS score because it is less prone to the influences of sedation and paralysis.

Secondary Insults

The injured brain is more vulnerable than a normal, healthy brain to systemic secondary insults such as hypoxia and hypotension. In the experimental but also in the clinical situation, the occurrence of secondary insults increases the degree of secondary damage after injury. The presence of secondary insults is associated with poorer outcome,^28,^56,⁵⁷ and the depth, duration, and number of hypotensive insults all cumulate toward poorer outcome.^20,^58,⁵⁹ Most studies have focused on early hypotensive and hypoxic events in which hypotension was defined as any episode with a systolic blood pressure lower than 90 mm Hg. The association between the actual blood pressure on admission and outcome has been further analyzed in a continuous manner by the IMPACT study group.²¹ These studies, which incorporated data from 6801 patients, showed that the relationship between blood pressure and outcome is continuous: low blood pressure and high blood pressure are both associated with a poorer outcome. After adjustment for age, motor score, and pupillary reactivity, the effects of higher blood pressure, however, largely disappeared, thus indicating that this association is most likely secondary to increasing severity of the injury. Various studies have shown that the combination of hypoxia and hypotension has a greater adverse effect on outcome than can be explained by either insult alone; the effects, however, appear to be subadditive rather than synergistic.

Structural Abnormalities

CT is the investigation of choice in the acute phase after TBI to identify the presence and extent of structural damage. The relevance of CT scanning for the purpose of classification and prediction has increased with the growing difficulties in reliable assessment of clinical severity according to the GCS.^54,^60,⁶¹ The prognostic value of individual CT characteristics in patients with TBI, including status of the basal cisterns, midline shift, the presence and type of intracranial lesions, and traumatic subarachnoid hemorrhage (tSAH), is well documented. In 1991, Marshall and associates introduced a descriptive system of CT classification (Table 340-7) that focuses on the presence or absence of a mass lesion and differentiates diffuse injuries by signs of increased intracranial pressure (ICP), such as compression of the basal cisterns and midline shift.⁶²

TABLE 340-7 Marshall Computed Tomographic Classification

STRUCTURAL DAMAGE	COMPUTED TOMOGRAPHY
Diffuse injury I	No visible pathology
Diffuse injury II	Cisterns present, midline shift of 0-5 mm, and/or lesion densities present or no mass lesion >25 cm³
Diffuse injury III (swelling)	Cisterns compressed or absent with a midline shift of 0-5 mm or no mass lesion >25 cm³
Diffuse injury IV (shift)	Midline shift >5 mm, no mass lesion >25 cm³
Evacuated mass lesion	Any lesion surgically evacuated
Nonevacuated mass lesion	High- or mixed-density lesion >25 cm³, not surgically evacuated

This classification is also strongly related to outcome, with the poorest prognosis in patients with CT category IV (signs of raised ICP plus shift) and the best outcome in patients without visible structural abnormalities. The Marshall CT classification, however, has limitations, such as the broad differentiation between diffuse injuries and mass lesions and the lack of specification of the type of mass lesion. Thus, this classification might mask signs of raised ICP in addition to a mass lesion and does not fully use the prognostic information contained in the individual CT characteristics scored. Maas and coworkers proposed a score chart for assessing the risk for poorer outcome based on individual CT characteristics and showed that such a chart results in better discrimination between patients with better versus poorer outcome than does the descriptive Marshall classification (Table 340-8).⁶³ This advantage was confirmed in subsequent work by Flint and colleagues.⁶⁴ The prognostic relevance of tSAH was extensively described by Kakarieka and associates after extensive analysis of the nimodipine studies. Later work confirmed the presence of tSAH as one of the strongest CT predictors of outcome after TBI.⁶⁵ Most studies, however, have concentrated on the presence or absence of tSAH without differentiation of the location (basal cisterns versus cortical) or extent. Cortical tSAH is frequently associated with underlying contusions, and its relevance is probably different from that of tSAH in the basal cisterns, which may incur an increased risk for vasospasm. More detailed analysis of tSAH and outcome after TBI should be a priority for the future.

TABLE 340-8 Rotterdam Prognostic Computed Tomographic Score

PREDICTOR VALUE	SCORE
Basal Cisterns
Normal	0
Compressed	1
Absent	2
Midline Shift
No shift or shift ≤5 mm	0
Shift >5 mm	1
Epidural Mass Lesion
Present	0
Absent	1
Intraventricular Blood or tSAH
Absent	0
Present	1
Sum score*	Σ + 1

tSAH, traumatic subarachnoid hemorrhage.

* The sum score can be used to obtain the predicted probability of mortality from the formula below. We chose to add plus 1 to make the grading numerically consistent with the grading of the motor score of the Glasgow Coma Scale and with the Marshall CT classification. The corresponding probabilities are calculated with the following formula: Probability (mortality) = 1/[1 + e^{−(−2.60+0.80×sum score)}].

Laboratory Parameters

Relatively few studies have investigated the relationship between laboratory parameters on admission and final outcome. This may seem surprising inasmuch as analysis of the prognostic value of laboratory parameters is particularly relevant because these parameters are routinely measured and objective. Abnormal values may also be modifiable. Various studies have shown a strong relationship between a poorer outcome and higher glucose values, low hemoglobin, low platelets, and coagulation disturbances. The results from the IMPACT studies have demonstrated that the addition of laboratory values to a prognostic model increases discrimination. These studies showed the greatest discriminatory properties for coagulation abnormalities and glucose (see Table 340-5). Although laboratory values may be modifiable, the observed association between abnormal values and poorer outcome, however, does not by definition mean that correcting these abnormal values will indeed improve outcome. The observed abnormality may simply be an expression or surrogate marker of the severity of injury. Prospective studies, preferably randomized controlled trials, are required to prove such an effect. Currently, there is great interest in various biomarkers released from damaged or necrotic neurons and glial cells in the brain. Various experimental and preliminary clinical studies have confirmed this potential.

Prognostic Models

Prediction Models

Prediction models provide diagnostic and prognostic probabilities and are therefore becoming an increasingly important tool in clinical medicine. From the perspective of prognosis after TBI, we may want to estimate—or predict—the risk for death of a certain patient admitted to the hospital with severe TBI. Single predictors often have insufficient predictive value to distinguish patients who will do well from those who will do poorly. Moreover, patients can have different characteristics that affect the prognosis in opposite directions. For example, for a 24-year-old patient with fixed pupils, we would predict a favorable outcome based on age but an unfavorable outcome based on pupil reactivity. Thus, estimation in prediction research is by definition a multivariable challenge in which multiple risk factors need to be considered jointly with multivariable analysis. For this purpose, relevant prognostic factors are combined in a prediction model and often presented as rules or nomograms. The most common approach to develop a prediction model is regression analysis. Alternative methods include recursive partitioning or classification and regression tree (CART) methods, as well as neural networks.

Usefulness

To be useful for physicians in clinical practice, a prediction model needs to meet stringent quality criteria. Valid development is important, with specific attention paid to prevention of overfitting. Overfitting means that the model described fits the study population very well but is unlikely to give reliable predictions for new patients. External validation should be considered mandatory.

FIGURE 340-2 Predicted probabilities of mortality and unfavorable outcome at 6 months after TBI in relation to the sum scores from the core, extended, and laboratory models. The logistic functions are plotted with 95% confidence intervals. Dot size is proportional to sample size. Sum scores can be obtained from Table 340-9. CT, computed tomography.

Various prediction models have been proposed for use in patients after TBI. Two systematic reviews, however, showed many shortcomings in model development and validation.^66,⁶⁷ The methodologic shortcomings in the development of prognostic models may be considered one of the major factors that have delayed more general acceptance of such models into clinical practice. Specific attention to the methodology of model development is therefore required. There are seven logically distinct steps in the development of valid prediction models with regression analysis:

1 Problem definition and data inspection: What is the research question? What is already known about the predictors? What is the outcome of interest? Are there missing values?

2 Coding of predictors: Do categorical and continuous variables require recoding? Dichotomization of a continuous predictor should be discouraged and a continuous approach to analysis preferred.

3 Model specification: What predictors should we include in view of what is known about the predictors already, and what is observed in the data under study? Stepwise selection methods are widely used but have many disadvantages, such as instability of the selection, bias in estimated regression coefficients, and underestimation of uncertainty in the selected model. The number of predictors that can be considered for inclusion in a prognostic model should be limited to prevent overfitting. As a rule of thumb, this number can be approximated by dividing the number of events (outcome) by a factor of 10.

4 Model estimation: Estimation of model parameters is commonly done by regression analysis. Some modern techniques have been developed that aim to limit overfitting of a model to the available data.

5 Model performance: For a proposed model, we need to determine its quality with measures for model calibration and discrimination. Calibration refers to the reliability of predictions: if we predict 10%, on average 10% of the subjects with this prediction are expected to experience the outcome. Discrimination refers to the ability of a prediction model to separate subjects with and without the outcome and can, for example, be quantified by the area under the receiver operating characteristic (ROC) curve. This curve shows the relationship between sensitivity and specificity. Overall model performance measures include the R² and the Brier score.

6 Model validity: Because overfitting is a central problem in prediction modeling, we need to consider the validity of our model for new subjects rather than for those in the data set used for model development. Several statistical techniques are available to evaluate the internal validity of a model (i.e., for the underlying population from which the data set was sampled). Internal validation may address statistical problems in the specification and estimation of a model (“reproducibility”). Common methods are cross-validation and bootstrap resampling procedures.

7 Model presentation: A final step to consider is the presentation of a prediction model. Regression formulas can be used, but many alternatives are possible for easier applicability of a model, including score charts, nomograms, and Web-based calculators.

When a valid model is developed, one of the next requirements is external validation of the model. Validation in multiple settings is required before application of a model can be recommended. Finally, we need to consider an impact analysis in which a prediction model is used as a decision rule and any improvement in physicians’ decisions is determined. Validation of a prediction model may indicate the efficacy of a rule (the maximum that can be attained with 100% adherence), but impact analysis will indicate its effectiveness in practice.

Recently, two prediction models, developed from large patient series and externally validated, have been published: models presented by the MRC CRASH trial collaborators ⁶⁸ and a prediction model proposed by the IMPACT study group.⁶⁹ The CRASH model also included patient data from low- and middle-income countries. Importantly, both models were developed from data available on admission, before provision of specialist care. They are therefore ideally suited for a baseline calculation of prognostic risk. Both models showed good performance in terms of both discrimination and calibration. Both approaches confirmed that the largest amount of prognostic information was contained in a core set of three predictors: age, motor score, and pupillary reactivity. The IMPACT study group further evaluated the additional benefit of adding more “building blocks,” such as structural imaging (CT characteristics), secondary insults, and laboratory data. Better performance was noted in a model that included this information.

The CRASH and IMPACT models are available on the Internet (www.crash2.lshtm.ac.uk/risk%20calculator/index.html and www.tbi-impact.org).

The IMPACT model is also presented as a simple score chart for sequential application of the models (Table 340-9). This score chart can be used to obtain an approximate prediction in individual patients. The predictive risk can then be derived by reading the predicted probability from nomograms.⁶⁹

TABLE 340-9 Score Chart for 6 Month Outcome after TBI