Ask

In contrast to the latter two cases described, Ms. Mason’s case represents true diagnostic uncertainty. She has a pretest probability (actual risk of having SAH at the moment of our encounter) that is neither immediately intuitive nor easily estimated based on clinical experience. It is in such scenarios that the medical literature is particularly useful. However, before seeking answers, we must be explicit in defining our question. We could start with a broad inquiry such as “What is the chance that she has SAH?” Formulating plus refining this question is a difficult and fundamental skill of EBM. It is an important endeavor that often has the greatest bearing on the value of the answer generated by the EBM process. Accordingly, considerable time and thought should be allocated to careful deliberation over what it is that we actually want to know. In this case, because we are primarily concerned with prognosis and diagnosis, we will focus most of our energy on defining the population most relevant to our patient.

The prevalence of SAH varies considerably across practice settings and research cohorts, which is often a reflection of biases such as spectrum bias, or enrollment of an overly narrow or broad spectrum of patients, and referral bias, which is one type of spectrum bias and typically occurs when a study enrolls only a referred patient cohort. A recent publication of patients referred to a neurosurgical department for headache suspicious of SAH found a remarkable prevalence of 59%.³ In contrast, in a large retrospective study of nearly all patients with headache encountered in an ED, only 1% were found to have SAH.⁴ In the second example, a broad cohort with limited inclusion or exclusion criteria related to headache quality was evaluated, and very few had the disease of interest. Conversely, when only a highly selected population of patients with headache referred to a neurosurgeon were studied, the majority had significant pathology. Ideally, any study that we use to generate a risk estimate for Ms. Mason should be from the ED setting and should enroll those with a similarly “acute” headache.

Further refining our question, we note that Ms. Mason is awake and alert with normal findings on neurologic examination. One review has suggested that 15% of patients seen in the ED with a thunderclap headache have SAH, but the two studies in this review did not distinguish between neurologically compromised and normal patients.^5,6 In both studies, one in five patients enrolled was noted to have either lethargy or altered mental status. Not surprisingly, another prospective study in which fewer patients with abnormal neurologic findings were enrolled found a much lower prevalence of disease⁷ (Table 1).

Incorporating this background research into our question, we decide that the following variables are critically important when considering her risk for SAH: she has no comorbid conditions, is seen in the ED setting, and has a headache of acute onset. She has normal mental status and neurologic findings.

The outcome of interest is SAH. There are effectively two “gold standard” tests available for identifying this disease. The presence of subarachnoid blood on unenhanced computed tomography (CT) is the most common means by which this diagnosis is made. Alternatively, the presence of blood or xanthochromia in cerebrospinal fluid, acquired via lumbar puncture (LP), in the presence of an aneurysm or arteriovenous malformation on cerebral angiography is considered diagnostic. In some studies in which patients are being evaluated for SAH, not all undergo these conventional tests, and clinical follow-up is used as a surrogate measure. In these cases, the assumption is that those with SAH, missed at the index visit, would experience significant neurologic decline or death during the follow-up period.

Thus, we generate a question using terminology that will assist in the next step of the EBM process. In neurologically normal patients seen in the ED with an acute headache, what is the risk (or prevalence) of SAH?

Acquire

We begin our literature search seeking high-quality research involving patients with headache in whom SAH is a diagnostic consideration. We decide to use the free PubMed database to explore the comprehensive MEDLINE database. When searching large databases such as MEDLINE, it is generally easiest to start by combining the most general terms relevant to the question and then systematically limiting the search. Accordingly, we start with the following MeSH terms (medical subject headings, the controlled vocabulary used by the National Library of Medicine to index articles): “headache,” 52,670 results; and “subarachnoid hemorrhage,” 18,364 results. By combining the two terms with the Boolean symbol “AND,” we are left with 1224 results. To further restrict the search, we add the following limits: English, human, all adult, core clinical journals. There are 118 results remaining, a reasonable number of abstracts to review individually.

One of the first papers found is a recent publication by Perry et al.: “High Risk Clinical Characteristics for Subarachnoid Haemorrhage in Patients with Acute Headache: Prospective Cohort Study.”⁹ Scanning the abstract, we decide that it meets many of our important inclusion criteria and thus embark on the third part of the EBM process.

Appraise

Two major steps are involved in any critical appraisal: assessing the internal and external validity (the likelihood that study results represent truth) of the study. A few key methodologic features favor the internal validity of this paper. First, it was performed prospectively. In the context of a diagnostic study, this usually means that clinical variables are recorded in real time before the diagnosis is known by the treating physicians. In a diagnostic study using a retrospective design, physicians (and commonly consultants) recording clinical variables in the medical chart are often unblinded to the ultimate diagnosis and outcome of the patient. This can easily lead to a biased (systematic deviations from the truth in quantitative research) interpretation and documentation of the signs and symptoms. Second, the outcome measures were well defined. The presence of SAH was determined by testing or clinical follow-up. A positive test was either in the form of CT or abnormal findings on analysis of cerebrospinal fluid with an aneurysm evident on angiography. Because it is neither practical nor ethical to expose every patient to these commonly used criterion standard tests (the test accepted for determining the diagnosis), rigorous clinical follow-up via structured telephone interview and review of coroner office records was pursued as another way of determining patient outcomes. Third, only 1% of patients were lost to follow-up.

In terms of external validity (generalizability of the study results to our patient), there are three key features. This study was done in the ED setting. Patients were enrolled only if they had high-risk headaches: either sudden onset or associated with syncope for a duration of less than 2 weeks. Finally, they had to be alert, defined as a GCS score of 15, and were excluded if focal neurologic deficits were present.

Almost 2000 patients were enrolled, which makes this the largest and most robust data set of high-risk headache patients in an ED setting. SAH was confirmed in 130 patients, so SAH was diagnosed in 6.5% of the cohort, which translates to about a 1 in 15 risk.

Apply

The internal and external validity of the study results seems sufficiently strong to apply the information to our patient Ms. Mason. Taking that number to the bedside, you and the patient decide that for a potentially devastating, fatal condition such as SAH, a 1 in 15 risk is substantial and necessitates further testing. A CT scan of the head is ordered.

The patient has an uneventful course in the ED. Her CT scan is read as normal by the radiologist. You return to the patient’s bedside to have another familiar discussion.

When the diagnosis of SAH is being pursued, the current recommended approach includes a CT scan, followed by LP when imaging is negative.¹⁰ Based on study design and disease prevalence, the sensitivity of CT has varied widely across research populations, between 82% and 100%, and this has been judged inadequate in the context of the disease.⁸ Although a recent investigation of CT for SAH reported a sensitivity approaching 100%, spectrum bias was evident because all the subjects were referred to a neurosurgical center for evaluation.³

Moving back to our appraisal step, we perform a more in-depth analysis of the study results of Perry et al. In the cohort there were 1999 subjects, and SAH was diagnosed in 130 of them. One hundred twenty-one of the 130 SAH cases were detected by CT, thus generating a sensitivity of 93%, whereas the remaining 9 cases were detected by LP. A potential limitation exists in that only 80% of the patients enrolled underwent CT and only 45% had LP performed, the criterion standard for SAH, but nearly all had extensive clinical follow-up. Because it is presumed that dangerous SAH would result in a deleterious neurologic event at some point, we decide that clinical follow-up is a good surrogate outcome (an outcome that can act as a valid replacement for the criterion or gold standard). All CT positives were considered true-positives, so we presume a specificity of 100% for the test. With this information available, we can calculate the negative likelihood ratio (LR) for a normal CT scan. The LR is a simple tool that calculates the probability of the disease based on the sensitivity and specificity of the test and the test result. In this case, with a negative CT result and 93% sensitivity and 100% specificity of CT, the calculation is 1 − sensitivity/specificity, or (1 − 0.93)/1. The negative LR in this case is therefore 0.07 (95% confidence interval, 0.04 to 0.13).

We decide to presume a worst-case scenario for the imaging test and choose an LR of 0.13, which represents the upper boundary of the confidence interval. Plotting a pretest probability of 6.5% on a Fagan nomogram and using an LR of 0.13, we generate a posttest probability of just under 1% (Fig. 1). You explain to the patient that a less than 1 in 100 chance (1 in 118 to be exact) persists that she has SAH detectable by LP. She carefully considers these numbers and asks another question: “How much lower would her risk be if LP were performed and determined to be normal?” In the EBM style of practice, one question often begets another. We return to the acquire and appraise steps.

Fig. 1 Plotting a pretest probability.

The aforementioned literature search yields a paper by Perry et al.: “Is the Combination of Negative Computed Tomography and Negative Lumbar Puncture Result Sufficient to Rule Out Subarachnoid Hemorrhage?”¹¹ In this study of 592 subjects, a subset of patients from the cohort described earlier by Perry et al., all patients with suspected SAH underwent CT and LP. Of those in whom both tests were negative, the posttest probability or risk of having SAH was 0.0001%, or 1 in a million.

You apply this information at the bedside. A negative LP finding would lower the patient’s risk for SAH from just under 1 in 100 to 1 in a million. The patient considers this information and seems conflicted. Both these risk estimates are quite low, although one clearly represents better odds. She does not want any invasive testing, however, and therefore probes further for information on the potential benefits of performing LP. She asks, “If SAH is detected by LP, what are the chances that she will benefit from that discovery?”

You reflect that this is an odd question to consider as an emergency physician. For diseases such as SAH, our goals have usually centered around detection, with a presumption of benefit to all in whom the diagnosis is made. Answering this question requires incorporating background knowledge about SAH with literature describing both the natural history of SAH and the absolute benefits associated with current neurosurgical interventions. We move to a place outside the official road map: background research involving reappraisal of the root literature. This is often an exciting place to venture and has been made more accessible by the recent efforts of many journals to electronically catalog older studies.

Any discussion of the potential benefits of detection of SAH must first start by distinguishing between aneurysmal and nonaneurysmal disease. Published case series of patients with SAH suggest that roughly 85% of cases of SAH can be attributed to leak or rupture of an aneurysm.¹² Of the 15% that are nonaneurysmal in origin, most are classified as “perimesencephalic” bleeding events in the absence of a vascular aneurysm. Small case series suggest that patients with perimesencephalic SAH have an almost universally good prognosis.^13,14 Of equal importance, no neurosurgical intervention is performed for these patients.

Applying this information, the benefits of diagnosis would probably be limited to (at the most) 85% of patients with SAH detected by LP. A patient’s post-CT risk for SAH of 1 in 118 should be modified to a 1 in 139 chance of potential benefit (1/118 × 0.85) with early detection.

This last set of figures makes another assumption: that all patients with aneurysmal SAH receive and benefit from neurosurgical intervention. However, few interventions procure a number needed to treat (NNT: the number of patients who need to be treated to achieve one good outcome) of “one.” Accordingly, it is likely that some patients, particularly those neurologically normal with SAH, will experience favorable outcomes irrespective of intervention. Ultimately, we need to know the absolute risk reduction (ARR: the absolute difference in the rate of bad outcomes between experimental and control groups) associated with neurosurgical interventions in neurologically normal patients with aneurysmal SAH to estimate the benefits of detection. The study design ideal to answer this therapy question is a randomized trial. Our search is redirected to find randomized trials comparing neurosurgery (in the form of aneurysm clipping or endovascular coiling) with no intervention in patients with SAH.

An extensive search yields several related papers reporting on the results of the Cooperate Aneurysm Study, a randomized trial of patients with intracranial aneurysms and SAH performed in the 1960s.¹⁵ There were four treatment arms: regulated bed rest, drug-induced hypotension with bed rest, ipsilateral common carotid artery ligation and bed rest, and intracranial surgery. As the authors of the study suggest, the group receiving regulated bed rest offered the best representation of the natural history of SAH resulting from a ruptured intracranial aneurysm.

Of the 187 patients ultimately analyzed in the bed rest group, the mortality at 6.5 years (the duration of follow-up) was 55.1%. For those classified as being in good neurologic condition, either symptom free or with minor symptoms such as headache, meningeal irritation, or diplopia, the mortality during this period was 47.4%. Most deaths were the result of aneurysm rebleeding, although some were attributed to direct effects of the initial bleeding and others to vasospasm. About 7% of the deaths were unrelated to the SAH event. To be conservative, however, we assume that half the patients with SAH who are in good neurologic condition have a reasonably good prognosis and that half will die of their disease. Thus, the potential for benefit via surgical intervention and modern therapies will be limited to about 50% of those in whom SAH is diagnosed because the others would have had good outcomes regardless of treatment.

In addition to improvements in neurosurgical interventions, medical advancements since the 1960s have resulted in many beneficial treatments for patients with SAH, including drugs such as calcium channel blockers.¹⁶ In one of the most recent, high-quality trials comparing endovascular treatment with standard neurosurgery, in which 88% of the patients were enrolled in good neurologic condition, 23.5% of those in the superior therapy arm, endovascular treatment, were dead or dependent at 1 year.¹⁷ A large, prospective observational study that enrolled more than 3500 patients with aneurysmal SAH reported similar survival data for patients initially admitted in good neurologic condition.¹⁸ Clearly, the contemporary cohorts fare better than patients with similar disease randomized to bed rest in the 1960s because 76% are alive with good function in some treatment groups, but it is critical to note that even with the best, most modern interventions, a substantial percentage of patients still have poor outcomes.

Accordingly, using the earlier numbers, it would seem that for neurologically normal patients in whom aneurysmal SAH is diagnosed, current neurosurgical and medical interventions are associated with a 30% ARR in mortality and poor outcome (50% to 23.5%), which translates to an NNT of about 3 (100/30). This indicates a tremendously powerful intervention inasmuch as very few modern therapies procure this likelihood of benefit.

We return to our previous calculations. The number of LPs needed to diagnose SAH was between 118 and 139. Multiplying this number by 3, we generate the number of LP tests that we would have to perform to benefit the patient with a neurosurgical procedure. This translates to a number between 354 and 417. Rounding off these numbers to facilitate communication, you explain to the patient that there is probably in the range of a 1 in 300 to 400 chance that she will be truly benefited by having LP performed.

The patient considers this information and decides to forgo the test. She understands that the disease has not been completely “ruled out” and that a negative LP would substantially lower her risk of having the disease.¹¹ In the medical chart you document that an extensive discussion regarding the risks and benefits of testing has transpired. You discharge the patient with a diagnosis of headache and tell her to return immediately should her symptoms return or any other concerning symptoms develop.

Background

Approximately 8 million patients seek treatment in the ED for chest pain each year, and these patients are often admitted for a work-up to rule out cardiac disease. It has been suggested that a large proportion of hospitalizations are unnecessary and that many patients undergo expensive and sometimes invasive testing.¹⁹ Conversely, studies also suggest that between 2.1% and 4.6% of patients with myocardial ischemia are initially discharged.²⁰ Groups that seem to be at highest risk for discharge during an initial coronary manifestation are women, nonwhites, and those with a nondiagnostic initial ECG.²¹ Given the significant morbidity and mortality associated with myocardial ischemia, it is crucial that patients at risk for poor outcomes be quickly and accurately identified. The challenge for the emergency physician is in accurately stratifying risk so that patients and physicians can make clinical decisions informed by valid and relevant evidence.

In this section we discuss a subset of studies within the medical literature that deal with prognosis. These studies assess the likelihood of specific outcomes for a given patient population and can provide useful data to guide clinical decision making.

Prognosis is dependent on a number of patient characteristics, which are referred to as prognostic factors—factors that affect the outcome of the disease. These factors are not necessarily the same as risk factors, which are characteristics associated with the development of a disease. For example, in the case of our patient, gender may be considered a risk factor for heart disease, but it is not necessarily associated with a specific outcome. In contrast, the patient’s normal ECG on arrival may be considered a prognostic factor if it is found to be associated with a better overall prognosis. In the case of cardiac disease, studies show that risk factors for cardiac disease are not generally useful in identifying acute coronary syndrome (ACS), except in the case of young patients, in whom the presence of four or more of these risk factors has been associated with a significantly increased likelihood of ACS.²²

In the following sections we will again use the ask, acquire, appraise, and apply technique to discuss prognostic studies with a focus on the risk stratification of “low-risk” patients with chest pain.

Ask

Again, the process of formulating and articulating a clinical question that will lead to the best and most useful answer for your patient scenario is complex, and it is important to remember that we are summarizing. In the case of prognosis studies, we must properly define the most relevant study population, and we must also precisely describe the outcomes that our patient is most interested in avoiding. In this case, the patient is a 45-year-old man with no past medical history, so the studies that we look for should be those involving adult chest pain populations without significant preexisting disease. The outcomes of greatest interest to a patient seeking medical counsel for chest pain are likely to be myocardial infarction (MI) and death, both outcomes that are fairly easily defined and tracked. Thus one way to formulate our question would be, “In adult patients seen in the emergency department with chest pain and a normal ECG, what is the 30-day risk for MI or death?” Studies best able to answer this question would be those in which a cohort of adult patients with chest pain whose initial ECG was normal are monitored prospectively and rates of death and MI are recorded.

Acquire and Appraise

One option to consider, particularly when attempting to acquire evidence in an active setting of patient care (when time restraints are important), is to defer to established sources of already appraised and summarized data. One such “preappraised resource” is BestBets, a website respected by many EBM users that offers brief and simple summaries of evidence-based reviews of topics. In this case a quick BestBets search for this topic reveals an article titled “The First ECG Has Low Sensitivity for Myocardial Infarction in Patients with Chest Pain” (2000).²³ This article examined 10 studies conducted between 1976 and 1998 that sought to assess the prognostic value of an ECG and reported sensitivities ranging from 13% to 69%. The article concluded that an initial ECG cannot be used to rule out MI. At first it seems that this review is helpful. However, on reconsideration, we realize that although information about the ECG may be generally useful, it is specific neither to our patient nor to our question, which is about prognosis. This review seems focused on the diagnostic performance (sensitivity) of a particular test (ECG).

We therefore move to performing a MEDLINE search, a fundamental skill set that is our next logical step when preappraised resources do not provide the answer. Our search, which focuses on the prognosis of patients with chest pain and low-risk features, reveals two potentially relevant and recent prospective studies. These studies distinguished between rates of MI and rates of death and had initial ECGs analyzed by physicians blinded to patient outcomes. ECGs were classified into one of a number of groups ranging from normal to acute ST-segment elevation MI. The prospective studies quoted mortality rates lower than 1% for patients with normal ECG findings.^19,24 These studies used slightly different cohorts, and the follow-up times also differed between studies. The study that is probably most relevant to our patient is the study by Forest et al., which involved 3814 adults older than 24 years who had an ECG performed for chest pain. This study found that 2% of patients with an initial normal ECG eventually suffered MI within 30 days [39 of 1912 patients], and 0.5% died [9 of 1912]²⁴ (Fig. 2).

Fig. 2 Relationship between the initial electrocardiogram and 30-day triple composite event rate (death, acute, myocardial infarction, and revascularization), as well as individual event rates.

AMI, acute myocardial infarction; Comps, complications; MI, myocardial infarction; Repol, Repolarization; Revasc, revascularization.

(Adapted from Forest RS, Shofer FS, Sease KL, et al. Assessment of the standardized reporting guidelines ECG classification system: the presenting ECG predicts 30-day outcomes. Ann Emerg Med 2004;44:206–12.)

The next step in your analysis might be to ask whether the patient’s prognosis would be altered by the inclusion of cardiac enzyme testing. In this patient with a somewhat concerning description of pain and no clear diagnosis, it would be reasonable to consider cardiac enzyme testing. However, if these tests are performed and come back negative, what is the next step? Should another set of enzyme testing be performed? Can your patient go home? This is a somewhat more complicated question to test than the ECG question because there are a variety of laboratory tests available and a variety of protocols for tracking cardiac enzymes. A MEDLINE search for “cardiac enzyme” or “troponin” along with “chest pain” and “prognosis” yields a number of studies that monitored outcomes in patients with negative initial enzymes. One of the most high quality of these was done by Hamm et al. in Germany: “Emergency Room Triage of Patients with Acute Chest Pain by Means of Rapid Testing for Cardiac Troponin T or Troponin I.”²⁵ Patients were enrolled if they had acute anterior, precordial, or left-sided chest pain for less than 12 hours. All patients had a troponin test done on arrival and 4 hours later, and those experiencing chest pain for less than 2 hours had another test sent at hour 6 (only 3% of subjects). This was a relatively high-risk cohort in which 30% were admitted to the intensive care setting. When troponin I testing was negative in the ED setting, the 30-day risk for death or MI was 0.3% (Table 2).

As we continue acquiring and appraising this body of literature, it becomes increasingly clear that a number of variables must be considered in making decisions about patients with chest pain. Thus it may be most useful to search for studies that seek to assign risk categories to patients based on combinations of clinical variables. Many studies have developed decision aids for assessing patients with chest pain, and although critical appraisal of decision aids is a complex skill set of its own, it may not be necessary to appraise the decision aids that these studies generate because there is also a great deal of simple, valuable information on prognosis in decision aid studies. Indeed, studies seeking to create or test decision aids often contain high-quality observational data that can be appraised independently from the decision aid being tested. Thus, we will use the rich database of observational information from decision aid studies of chest pain to find information that is relevant and useful to our patient.

The Thrombolysis in Myocardial Infarction [TIMI] risk score has been tested in multiple populations, and most studies of chest pain patients assign a TIMI risk score to subjects. This score (see Box 1) uses seven prognostic factors, with each assigned 1 point, to create an overall risk score. The score was originally derived by using a population of patients with unstable angina or non–ST elevation MI and was used successfully to risk-stratify these patients.²⁶ It was then externally validated in a population of patients with undifferentiated chest pain.²⁷ Thus the score seems reproducible, valid, and applicable to a variety of populations. For these reasons, virtually all decision aid studies of patients with chest pain classify patients by TIMI risk score. Therefore we can quickly and easily calculate the TIMI risk score of our patient (it is 0) and then look at these observational data to see what the risk for MI or death was for study patients with the same risk score. Unfortunately, not all studies reported rates of MI or death separately but instead reported them in a composite outcome, including the end point, revascularization. From an EBM perspective, this last end point is problematic for two reasons. First, a high degree of subjectivity enters into decision making about which patients undergo revascularization, and second, its utility in patients not actively experiencing an MI is dubious, thus limiting its function as a “patient-important outcome.”²⁸ There are four studies that reported MI, death, and revascularization rates separately, two of which actually used a modified TIMI scoring system in which subjects were classified as low, moderate, or high, with low representing a score of 2 or less^27,29–31 (Table 3). With approximately 1600 subjects across these studies, for patients with a TIMI risk score of 0 or low, the reported rates of MI and death at 30 days were 0% to 2% and 0% to 0.2%, respectively. For 400 subjects with a TIMI score of 0 and an alternative diagnosis per treating emergency physician, the death rate was 0.8%.²⁹ Translating this into understandable numerical values, in patients classified as low risk by TIMI scoring, the 30-day risk for MI is lower than 1 in 50, whereas the risk for death is less than 1 in 500.

Apply

Clearly, identifying patients with “low-risk chest pain” is difficult, and many factors need to be considered, including the history and physical examination, ECG, and the results of laboratory testing. In the case of our patient, how does the evidence that we have just discussed help guide our management? The evidence suggests that although his normal ECG is certainly reassuring and helps classify him as low risk, his symptoms mean that he does have a risk for MI or death that is calculable and real. The presence of two negative sets of cardiac enzymes may further improve his prognosis (and is necessary for him to remain in the TIMI risk category of 0) but still does not reduce his risk to a zero chance for MI or death.

Ultimately, the data presented here are most useful for initiating an informed discussion with our patient. What level of risk is he willing to accept, and how does this relate to the level of risk you would accept? For some patients, a 1% to 2% risk for MI or death at 30 days may be acceptable because they may plan to follow up with a primary care doctor in that time who knows them well and they may dislike the prospect of an inpatient stay or work-up enough to take on some risk. Other patients, at the same risk level, may strongly desire further testing, including another set of cardiac enzymes and a potential inpatient stay. As in all aspects of medical care, clinical decisions must ultimately be based not only on knowledge and evidence but also on the expectations, desires, and values of the individual patient. As evidence-based physicians our job is to act as counsel to our patients and advocate for their interests by providing them with the information that they need to make informed, reasonable decisions. In this case we discuss with our patient the risk for MI and death and the nature and potential value of an inpatient stay and further testing. He is interested and engaged in the conversation, understands the potential risks and benefits, and decides that he would prefer to go home to his family and see his primary care physician tomorrow. We arrange the appointment for him, explain what to watch and return for, and wish him well.