Success Criteria and Reported Efficacy Rates

A historically accepted success criterion in neurostimulation studies for pain is that greater than or equal to 50% of individual patients should report greater than or equal to 50% pain relief (PPR; or ≥50% reduction in the visual analog scale [VAS] of pain intensity) at follow-up—commonly after 6 to 24 months. Until the 1990s, most trials and studies had used minor variations of the 50 : 50 standard.^6,47 By definition, and to prevent a few cases from skewing the results, average or aggregate pain ratings formerly were avoided in efficacy rate calculations. After historical series failed to achieve the 50 : 50 efficacy criterion, studies of SCS between the 1990s and the present were structured to determine whether statistically significant aggregate or average pain relief scores differed after treatment over relatively short periods (<2 years) compared with the patient’s baseline. According to our previous analyses and data summarized in Table 165-2, assessments commonly were performed by the implanters or their associates or did not reach the 50 : 50 efficacy threshold, or both. Studies of novel programming techniques or lead designs also found limited efficacy or lacked sufficient data to draw long-term conclusions. Most of the studies that we reviewed had open-label designs, and all compared the results with the patient’s pretreatment baseline. None reported greater than or equal to 50% long-term relief in greater than or equal to 50% of subjects unless efficacy was analyzed without regard for the duration of follow-up or unless subjects who were lost or disqualified from follow-up were excluded from the calculations.

Long-term, prospective, randomized, crossover comparisons of SCS and reoperation in subjects with persistent pain after previous back surgery, performed over about a 10-year interval, were not designed to measure the degree of pain relief afforded by SCS. Despite the preference of subjects for SCS compared with reoperation, the effect size of the analgesia afforded by SCS or of repeat low back surgery remained undefined.^64,66 A problem with such study designs, in addition to the assignment of one cohort to repeat low back surgery, a treatment that had previously not relieved their pain, is that subjects with a demonstrable anatomic cause for their pain were as likely to be assigned to SCS as to another low back operation. Likewise, subjects with no demonstrable spinal abnormalities had a 50% likelihood of being assigned to undergo another spine operation despite the absence of surgically amenable pathology. Neither treatment served as an adequate control measure for the other, leaving the study uncontrolled, unblinded, and unlikely to answer fundamental questions about the efficacy of SCS.⁶⁷ One randomized controlled trial, among nearly 600 articles on SCS for RSD or CRPS, found that SCS plus physical therapy (PT) significantly reduced pain after 6 months and 2 years compared with PT alone, but not after 5 years.^53–5563 The early results did not achieve the traditional 50 : 50 success criterion, and the long-term results revealed no statistical difference between the SCS plus PT group and the PT alone group.

In all DBS clinical series reported until recently, fewer patients reported pain relief after 6 to 24 months of therapy than during the early postimplantation period, and no study achieved the 50 : 50 success level unless short-term results carried the same analytical weight as follow-up that lasted months to years.^6,5,8 Two industry-sponsored open-label DBS trials for pain enrolled 246 patients from 1989 to 1995. The coated-wire DBS lead used in the first trial became obsolete and was withdrawn from the market (N = 196); a trial of the current model DBS lead was closed because of slow enrollment and unexpectedly low efficacy (N = 50).⁶ Nandi and associates^83–85 originally reported that seven patients with internalized DBS systems (among eight patients with implanted electrodes) experienced 32% to 46% pain relief after 3 to 30 months. Subsequent reports from the same group described better results than the first cohorts. However, it eventually became impossible for reviewers to follow patients through the entire series of publications.^86–88 In another recent publication, investigators, one of whom had decades of experience in the field, described the results of a small 10-year case series that divided patients retrospectively according to diagnosis, and in which more than half of patients experienced no clinically meaningful pain relief at any time after DBS lead implantation.⁹⁰

Articles and reviews by investigators in Milan (8 of 14 reviewed in Table 165-2), nearly all covering the same overlapping patient cohort at different stages of follow-up, and all describing dramatic efficacy, have dominated the discussion of hypothalamic DBS to treat cluster headache and other intractable trigeminal autonomic cephalgias. The experience at other centers has been less dramatic, and a proposed U.S. trial to examine hypotheses generated during a physician-sponsored pilot study may eventually achieve funding-agency approval.¹⁰⁶ However, a randomized, controlled, and blinded French multicenter trial recently failed to demonstrate a statistically significant difference in the prespecified primary efficacy variable (the weekly frequency cluster headache attacks) between the stimulation “on” and “off” periods.¹⁰⁷

The industry-sponsored European multinational study of MCS to treat facial (trigeminal) deafferentation pain and central poststroke pain enrolled and implanted only 24 subjects in 2 years, out of a planned cohort of 104 enrollments calculated to yield 82 evaluable subjects for the primary efficacy end point, before being closed by the sponsor because of slow enrollment. Although one can draw only limited conclusions because the prospectively planned N number was not achieved, 7 of 24 subjects (29%) withdrew or were discontinued from the study after lead implantation, but before randomization (to sham or active MCS) because of lack of efficacy during either the trial stimulation or early postinternalization study phases. Among the 11 randomized subjects who completed blinded one-way crossover MCS (on to off, or off to on; 4 weeks duration, each) and who completed all scheduled follow-up visits, none expressed a preference for the MCS “on” condition regardless of the on-off sequence to which they were randomized. No other large-scale blinded and controlled trial of MCS has been undertaken to date. Results of the MCS trial, albeit incomplete, fail to validate the more positive findings of the other controlled or uncontrolled single-center studies that we reviewed in Table 165-2.

Success rates in two early case series of ONS for head pain disorders were 64.5% and 100% during a mean follow-up of 18 months.^38,39 In one report of 25 patients, 9 (36%) still had moderately or severely disabling headaches after 9 to 36 months.³⁹ However, results of the randomized, controlled, industry-sponsored ONS clinical feasibility trial, which met prospectively defined enrollment, follow-up, and analysis goals (N = 110 enrolled subjects), showed no clear efficacy signal.¹⁵⁰ The difference in the primary efficacy end point “number of headache days per month” was not statistically significant among the adjustable stimulation group (active therapy) and three different control groups: a group with preset stimulation (1 minute of stimulation per day, automatically programmed), a group with medical management (no device implant), or an ancillary group of eight subjects who were implanted and received adjustable stimulation, but who had failed to respond to an occipital nerve block during the preimplantation screening phase. The positive results reported from other case series and studies analyzed in Table 165-2 do not comport with the findings of this monitored and audited industry-sponsored trial.

Nature of the Reports

Evidence-related features of the recent neurostimulation clinical literature are listed in detail in Table 165-2 and organized according to published criteria for the evaluation of medical evidence.^41,42 The prospective studies of SCS in RSD and of SCS compared with reoperation for low back pain each employed control groups and randomization. However, as noted previously, patients and evaluators were not blinded.^{53–55,63–67} Industry-sponsored studies of MCS and ONS were controlled, randomized, and blinded and revealed no clear signals of efficacy. This leaves unexplained the one prospective and partially blinded single-institution study of MCS that reported positive results.¹⁴⁵ The remainder of neurostimulation publications consisted of retrospective case series, case reports, or meeting abstracts (see Table 165-2).

Diagnoses and Patient Selection

Diagnoses, demographic data, the number of eligible patients who were evaluated or excluded, the pretreatment duration of pain, and previous pain therapies were reported unevenly or not at all in several publications (see Table 165-2). The assignment of diagnoses or pain categories, including nociceptive, deafferentation, central, mixed pain, and headache disorders, was inconsistent among centers, and sometimes among multiple reports from the same institution. Patients treated successfully with ONS for occipital neuralgia in one series were followed by a second series that had the newly minted diagnosis of chronic or transformed migraine headache using criteria that continued to evolve after commencement of the ONS trials.^38,149,155

Reports and reviews continued to describe selection criteria believed to maximize the likelihood of success for neurostimulation therapies, including pain in contiguous regions that can be covered by stimulation-induced paresthesias, favorable responses to transcutaneous electrical nerve stimulation (TENS), diagnoses previously reported to respond favorably, and rejection of patients with presumably unfavorable psychological or personality profiles, drug addiction, or pending litigation. In other reports, the results of psychological or pharmacologic screening either were not reported or had little influence on whether patients were accepted for therapy (see Table 165-2). The recently completed ONS feasibility trial revealed no predictive value for occipital nerve blocks. Psychological selection criteria for other neurostimulation therapies also had little predictive value.^5,156,157

Stimulation Targets, Test Simulation, and Ancillary Treatments

The neural target for SCS was the dorsal columns of the spinal cord at cervical or thoracic levels to treat upper or lower extremity pain, respectively, and using percutaneous or surgically implanted epidural leads. DBS targets for classic indications included the VPL-VPM thalamic nuclei or the PAG-PVG region, or both, depending on the surgeon’s judgment, the patient’s symptoms, or the results of preoperative screening tests. Hypothalamic target coordinates for DBS to treat cluster headache are very close to the classic PVG target.¹⁵⁸ MCS lead implantation techniques evolved during the past 20 years to include neuroimaging guidance, electrophysiologic localization, single-stage surgery, and epidural, instead of subdural, lead placement.^5,9 ONS involved percutaneous fluoroscopically guided insertion of SCS leads into the subcutaneous tissue at the C1 level. A transverse trajectory placed the electrical contact surfaces across the course of the greater and lesser occipital nerves. Technical reports and small case series described procedures for ONS lead placement under direct vision; however, those techniques have not yet been studied in formal clinical trials. For all neurostimulation therapies, the use or duration of a trial period before system internalization, the methods used to select active electrode contacts, the stimulation parameters that were employed, and the daily schedule of treatment sessions varied among centers and among individual patients. Many publications contained no information about how analgesic medications, psychoactive drugs, or other concomitant therapies were managed, and all information regarding decreased pain medication intake relied on patient self-reports that were not verified by pill counts, drug testing of body fluids, or other quantitative methods (see Table 165-2).

Follow-up, Outcome Measures, Data Collection, Reporting, and Analysis

Other than the industry-sponsored trials, relatively few studies employed independent evaluators (see Table 165-2). In most studies, the implanters or their associates, who were aware of the patients’ history and previous assessments and who were responsible for their ongoing care, performed follow-up evaluations. Again, with the exception of industry studies, the duration of follow-up often was unclear, not reported, or varied from months to years in the same series. And multiple reports of the same patient cohort frequently did not track individual cases through the series of publications. Because follow-up periods were not uniform, because subjects frequently did not comply with long-term follow-up, and because life table analyses were not performed, different durations of therapy carried equal weight in efficacy calculations. The studies that had well-defined follow-up periods are listed in Table 165-2 as “uniform timing reported” and included all of the recent and historical industry-sponsored trials.

Methods to assess efficacy included the VAS, PPR, and other standardized or nonstandardized scales and categories applicable to particular diagnoses (e.g., migraine or cluster headache). The two DBS trials that ended in 1995 were the last to require that as a criterion for success greater than or equal to 50% of the internalized patients had to report greater than or equal to 50% pain relief after a prospectively defined interval of 1 to 2 years. Subsequent reports of neurostimulation efficacy either sought statistically significant differences in aggregate pain relief scores (PPR or VAS) compared with baseline values or used descriptive outcome categories that spanned the 50% PPR or VAS value.

Reviews, Meta-analyses, and Practice Guidelines

Few reviews, editorials, or journal correspondence critically analyzed efficacy claims. As a rule, review articles recapped previous claims without analyzing the validity of the underlying methods and data, or the plausibility of the claims themselves. This held true for articles, chapters, meta-analyses, and quasi-official publications of evidence-based practice guidelines for neurostimulation therapies.^13,159–166 Most reviewers accepted the hypothesis that neurostimulation provided pain relief as described in the original reports, and some put forth mechanisms to explain stimulation-produced analgesia in the absence of evidence that analgesia actually occurs.¹⁶⁷

A shift away from prospectively defined success thresholds (such as the 50 : 50 rule) in favor of statistical end points, namely, significant differences between baseline and posttreatment pain scores, has made structured reviews more difficult, especially when publications contain incomplete data. Statistical difference–based end points also have prompted investigators and reviewers to perform multivariate analyses of apparent correlations between statistically defined efficacy and underlying patient characteristics—a practice of questionable statistical validity.

Diagnoses and Prognostic Factors

Patient selection is uniformly reported to influence the success of neurostimulation therapies. One fundamental selection criterion is the diagnosis or etiology of a patient’s pain. Table 165-2 indicates that individual diagnoses and broader diagnostic categories sometimes have been arbitrary or malleable. A few recent trials, which also showed no analgesic or salutary physiologic effects, attempted to narrowly specify diagnoses in the study protocols (e.g., SCS for angina pectoris, MCS for central poststroke pain) and to require documentation of the diagnosis in each subject’s medical record (e.g., imaging studies, electrophysiologic tests). Both factors, diagnostic consistency and assurance that each subject’s diagnosis has a sound anatomic and physiologic basis, are crucial to the successful investigation of any therapy for chronic pain.

Only small unblinded series supported selection based on responses to analgesic or sedative drugs or to local anesthetic injections. The investigators’ conclusions most likely resulted from patient suggestibility or from misattribution of predictive value to random fluctuations within small data sets. Other selection criteria have included the subjects’ past medical history and demographic and psychosocial factors. These were investigated in studies of SCS and were reviewed in the DBS literature, with disparate or inconclusive findings.^155,156 The most important limitation of previous analyses is that the influence of putative prognostic factors (e.g., the results of psychological or personality tests) could not be measured after those same factors were employed as inclusion or exclusion criteria in the same studies.

Trial Stimulation and Blinded Programming

Temporary stimulation with externalized leads allows patients to experience stimulation-induced effects, allows physicians to adjust parameters to minimize unpleasant side effects, and can help to avoid further investment in a therapy that may not match a patient’s analgesic needs or expectations. In actual clinical practice, trial stimulation or intensive postimplantation programming can influence patients to respond in an affirmative manner, especially if they are instructed to identify the settings that provide the best pain relief from among the choices presented to them. This phenomenon may explain in part why the proportion of patients who experienced long-term relief after an initial trial for the same diagnoses in different series has varied from near 0% to near 100%. One report described attempts to salvage postimplantation MCS analgesic efficacy by hospitalization of patients for intensive device reprogramming when they ceased to report analgesic efficacy after initial success.¹¹⁵ The entire series was not published, and it is plausible to speculate that at least some of the patients provided affirmative responses simply to escape from the hospital. Patient and evaluator blinding and the random or repeated presentation of different stimulation settings by a neutral individual and at prespecified or uniform intervals can reduce the occurrence and impact of unintentional coercion, cues, or suggestions.

Clinical Trial Design

We have proposed that new data would be required to investigate the efficacy of neurostimulation for pain because mining or meta-analyses of the available data could not repair deficits in the existing data and knowledge bases, nor provide accurate answers. New clinical investigations would take into account the methods for evaluation of medical evidence summarized in Table 165-1 and should include as many features as feasible from Table 165-3. Even formal compliance with level I criteria, namely, randomization of subjects and the presence of a control group, may yield data subject to significant limitations. Unintentional study design loopholes can include lack of blinding, ineffective blinding, and the selection of qualitatively different treatments for the active treatment compared with control arms (implant compared with no implant, or substantially different treatments). We recently discussed these issues and related ones in detail in a publication available online as full text.⁵

TABLE 165-3 Optimal Clinical Trial Design Features to Investigate the Efficacy of Neurostimulation

We applied clinical trial evaluation criteria to reports and reviews that were published during the past 15 years. Sound methods for experimental design, analysis, and reporting (including neurostimulation studies) have been available for many years longer than that. The finding that most recent publications and physician-initiated trials do not meet the methodologic and evidentiary standards now in force in the medical device industry means that the efforts and participation of investigators and subjects in such studies have been squandered. The questions and hypotheses generated by analyses of existing data could be answered by new data collected using optimized investigational methods outlined in this chapter and primary reference sources. However, as perilous as it may sound to advocate not asking or answering additional questions about the efficacy of neurostimulation to treat chronic noncancer pain, industry and physician sponsors may consider taking a “time out.” Reasons include a recent shift in the burden of proof; absence of physiologic, theoretical, or animal models that accurately and reliably translate to clinical results; poor understanding of how social, economic, and cognitive factors influence patient or subject responses; absence of will or resources among industry and physician sponsors to conduct studies of sufficient duration; and the problematic situation of how to craft informed consent for new clinical studies in the absence of a promising scientific breakthrough.

After so many decades of neurostimulation therapies not providing useful analgesic benefits, and after the recent failure of every well-designed trial to reveal useful benefits or to substantiate theoretical claims, advocates for such therapies now bear the burden of proof. In the absence of genuinely new science, targets, and techniques, the likelihood appears negligible that further investigations of existing therapies for new or existing indications will prove that stimulation-produced analgesia, as we now know it, really exists. One example of breakthrough basic science could include the development of new chronic pain models in experimental animals or the development of new and more useful methods to study and measure pain in existing models—or determination of whether what we interpret as pain in animal models translates to humans. In the absence of breakthrough science, and because so many recent well-designed trials have failed, the informed consent process to enroll subjects in additional trials of already-failed therapies will be problematic, at best.

We briefly have outlined how the growth of pain medicine, practiced mostly by physicians who are not neurosurgeons, intersects with the social and economic trend to shift work-related benefits off unemployment rolls and onto disability rolls. Disability entails medicalization and physician participation. Because documentation and advocacy alone are unrewarding for physicians financially and professionally, the market for billable interventions grew considerably as disability overtook unemployment over the past three decades. Neurosurgeons are trained to develop a suitably conservative and skeptical approach to patient selection for surgically treatable neurological disorders. Non–surgically trained physicians have a relatively unselective approach to the application of so-called minimally invasive therapies to chronic pain patients. Nearly all chronic pain patients who consult such practices are administered diagnostic, palliative, or therapeutic needle, catheter, or implant-based procedures. Procedure-based physician and facility fees thus make it possible to provide, prescribe, or approve ancillary benefits that are so important to the chronic pain patient’s economic survival. Sponsors and investigators’ lack of understanding of this situation, which has nothing to do with placebo effects, complicates all assessments of analgesic efficacy. Recent experience with trials that could not be carried to completion raises the related issue that it may no longer be possible to recruit a sufficient number of compliant patients or equipositioned investigators to execute well-designed studies. The lifelong duration of chronic pain makes it only reasonable to demand long-term evidence of efficacy—at minimum, 2 to 5 years. Because all comparably long studies or trials of neurostimulation to treat pain have failed, it is unlikely that sponsors have much of an appetite to risk failure for new indications or loss of existing broad regulatory approvals by conducting studies with meaningfully long-term efficacy end points.

In conclusion, extraordinarily durable biases on the part of investigators, a positive reporting bias in the clinical literature, and the nature of chronic pain have created a broad gulf between the expectations of patients and physicians and less optimistic findings of structured analyses of the clinical literature. One readily passes off expressions of marginally rational ideas about health and well-being among the lay public, a commonly observed phenomenon in social and cognitive psychological studies (e.g., positive attitudes prevent recurrence of breast cancer), as the product of uninformed minds.¹⁷⁶ In the case of physicians and other professional elites, highly educated individuals who should know better, the neurophysiological basis for the persistence of demonstrably false ideas is just now being elucidated.^177–180