Chapter 120 Brain–Computer Interfacing Prospects and Technical Aspects
It is possible that the brain–computer interface (BCI) has origins in this type of thinking. Thus, there is a logical extension of an electrical organ to control electrical devices. A BCI, sometimes called a brain–machine interface (BMI), is a thought translation device for direct communication pathways between a brain and an external device. We know the brain’s electrical activities are capable of pattern changes that can be actively conditioned. A simple example is production of alpha wave activity during meditation. The key problem with BCI is the neural interface. Neural signals need to be accurately detected and translated into useful command signals to effect control over computers or prostheses. The difference between BCIs and neuroprosthetics is the direction of the connection and this is increasingly blurred by feedback mechanisms. The terms can be used interchangeably, and both endeavor to achieve the same aims, such as restoring sight, hearing, movement, communication, or cognitive function. In many ways it is best described broadly as neuromodulation, the process of augmenting, inhibiting, modifying, or regulating the electrical or chemical neural interface to achieve therapeutic effect (Table 120-1). Neuromodulation is thus a very broad field covering every body function controlled by the central nervous system and embracing multiple clinical specialties. For the focus of this chapter we will restrict our discussion to classical BCI systems with signal acquisition, feature extraction, translation algorithm, and operational effect. The question is no longer whether damaged or degenerative brains can provide a foundation for BCI, as this is established, but rather how efficient can we make it?
Table 120-1 Neuromodulation: Merging Human and Machine
| Motor | Sensory | Disease |
|---|---|---|
|
|
BCI, brain–computer interface; DBS, deep brain stimulator; SC, spinal cord; VNS, vagus nerve stimulator.
The BCI development began by working backwards. Throughout the century, stimulation of the brain revealed fascinating aspects of how machines could control brain activities. Classic demonstrations of this were developed by Jose Delgado who was able to stop a charging bull with electric stimulation at specific sites within the brain.1 The stimulation gave the illusion of behavioral control. A practical extension of this developed into the deep brain stimulator (DBS), which is capable of controlling a variety of abnormal movements and is beginning to be applied to epilepsy, pain, psychiatric conditions, and numerous other diseases.2 Specific research on BCIs commenced in the 1970s at the University of California-Los Angeles and the expression “brain–computer interface” first appeared in scientific literature.3
The earliest clinical work suggested the possibility of neuromodulation by indirect means using noninvasive electromagnetic interfaces that are dominated by two “write-only” techniques—transcranial magnetic stimulation (TMS) and direct current stimulation (DCS)—and three “read-only” techniques—electroencephalography (EEG), magnetoencephalography (MEG), and functional magnetic resonance imaging (fMRI). There is no “read-write” noninvasive technology, but there is invasive technology capable of sensing, analyzing, and stimulating for effect. EEG is the most extensively studied BCI noninvasive interface (Fig. 120-1). In the 1980s, Farwell and Donchin4 developed a scalp EEG-based BCI using P300 (an EEG-evoked potential) signals to allow normal subjects to communicate words to a computer and thereby “speak” through a computer-driven speech synthesizer. The analyses suggest that this communication channel can be operated accurately at the rate of about 2.3 characters per minute. Speed can be increased by using paradigm matrixes (5 to 7 characters per minute), but severely impaired patients may not do well due to reduced attention span and other cognitive difficulties.5 A similar device allows volunteers wearing virtual reality helmets to use their P300 EEG signals to control elements in a virtual world including turning lights on and off.6
In the 1990s, Niels Birbaumer used EEG recordings of slow cortical potential to give paralyzed patients limited control over a computer cursor to write responses.7 Patients were trained to move a computer cursor by controlling their EEG activity. The training often took many months and the communication process was slow, requiring more than an hour for patients to write 100 characters. In 1998, Peckham and colleagues8 used a 64-electrode EEG skullcap to return limited hand movements to a quadriplegic. By concentrating on simple but opposite movements like up and down, the beta-rhythm EEG output was analyzed to identify patterns in the noise. Above average activity was set to “on” and below average to “off,” thus enabling the subject to control a computer that drives the nerve controllers embedded in his hands, restoring some movement. The essential step in all EEG BCIs is “recognition” of a specific control signal extracted from the Fourier components of the signal and identification of frequency bands related to each control action for each subject. The relatively long measurement period needed to obtain the low-frequency signatures imposes a significant limitation on how rapidly EEG BCI can transfer information. This approach requires a large computational investment of microprocessor arrays. The results are promising by providing communications conduits for severely disabled individuals. The accuracies of information transfer varying among individuals range from 65% to 98% (average 88%). Thus, there remain substantial barriers to using EEG as an effective and efficient controller of BCI.
Invasive or direct BCIs use probes that are implanted directly into the gray matter of the brain to produce the highest-quality signals in real time from areas of a few microns. Changes in spike frequency can be detected very rapidly. The first direct human BCI was an outgrowth of work on primates over a long period of time and demonstrated the ability of single motor cortex neurons to change their firing pattern in a plastic manner.9–11 Phillip Kennedy patented the use of a glass cone electrode filled with neural tissue to promote growth of local neurons into the electrode for recording. Kennedy and Bakay built the first cortical BCI by implanting neurotrophic electrodes into monkeys who could then quickly learn to voluntarily control the firing rates of individual and multiple neurons in the primary motor cortex by rewarding the generation of appropriate patterns of neural activity. The ability to record signals over a long period of time from the same neuron suggested that the activity of an individual neuron could be reliably used to control machinery. The first clinical approach was to restore communication to patients.12,13 The obvious machine to control was a computer, for through the computer it is possible to communicate with the world. The first paradigm was an on–off binary response (Fig. 120-2). The intention was to communicate by thought alone. The next endeavor was to learn to control individual units so that the patient was able to move a cursor across a computer screen and stop at the appropriate computer icons to produce computer-generated phrases such as “Hello, my name is JR.” The next step was to move in two directions and use the computer as a virtual typewriter to produce short responses by stopping on letters or punctuation icons (Fig. 120-3). This for the first time demonstrated unequivocally that cortical neurons engaged by movement intentions persist in the motor cortex years after disease or injury to the motor system, and that neuronal spikes and field potentials derived from motor cortex can be used by paralyzed patients to operate BCI. Thus movement programming remains. Also for the first time in humans, it was demonstrated that cortical motor neurons could learn non-motor tasks. Following this success, a company, Neural Signal, Inc. (Atlanta, GA), was created.
The second major clinical long-term study of patients with direct BCI is being conducted by Cyberkinetics Neurotechnology Systems, Inc. (Foxborough, MA). Similar to the neural signal background, this work grew out of primate studies.14–16 The procedures are very analogous to neural signal procedures, although the electrode is very different.17,18 The electrode uses 100 tines, and because of that the amount of information needed to be transferred is entirely too large to go through a radiofrequency transmitter; instead, the information is transferred directly to a head post and subsequently connected to the electronics. There are a variety of computer tasks that can be performed and even simple demonstrations of robotic movements (Fig. 120-4). Although the signals are theoretically stable, the initial patients have failed to demonstrate that a large number of signals can be maintained for more than 6 months without requiring intervention. In addition, there is a setup time required every day to evaluate which units are still active and able to be engaged. There is then an additional retraining time required. Although both BCI systems are extremely sophisticated, utility is still quite limited.19
Patient Selection
Because of the experimental nature of the procedure, the patients have been limited to intransigent medical conditions and thus must have severe fixed and/or progressive deficits. This has included patients with high cervical spinal cord injury, stroke (especially brain-stem stroke), cerebral palsy, and amyotrophic lateral sclerosis (ALS), but other diseases may qualify. The risks of surgery must be small to ensure benefit for the patient. Ethical issues are critical and need to be carefully addressed.20 This is a very vulnerable population who will accept high risks for the chance of minimal gain. An informed caregiver should be an integral part of the informed consent process.
The key to success of any surgical procedure is to carefully design the inclusion and exclusion criteria. There is always a temptation to include exceptional patients. This should be avoided. Inclusion criteria should reflect the patient’s primary needs and not stretch the limits of what is achievable. Thus, for a spinal-cord-injured (SCI) patient, it would be control over a motorized wheelchair, whereas for an amputee it would be fine motor control over a prosthetic device. Current interventions are for patients with severe disability. In the case of ALS, the clinical history is well recognized and obtaining permission for surgery before severe deficits occur could be an ideal situation to advance knowledge of what is possible for these patients. After severe deficits ensue, it will be harder to train and regain function. Similarly, although the stroke or SCI patient may need to have a high degree of disability, some residual motor activity, even if nonfunctional, could be extremely useful and might be essential for optimal function of BCI. Plasticity is necessary and any residual motor function may suggest that the circuitry is still readily available for the BCI. Again, even minimal residual sensory feedback may be ideal for advancing the learning curve and the development of analogs. Obviously, the brain must have some functional activity in order to be useful and an fMRI may be necessary to identify functional activity. These patients are generally quite sick and may require interventions of various kinds unrelated to the BCI surgery. These setbacks can be quite serious and the selection process should avoid patients who are metabolically unstable, and in general, are marginally fit for surgical intervention. Communication with these patients is essential and some form of communication has to be distinctly demonstrated. The more robust the communication, the better off the investigator and the patients are as they proceed in the trial.
The rationale for exclusion criteria need to be carefully considered. These must reflect the limits of both the device, as well as the limits of surgery. Exclusion criteria are: (1) medically unstable and unable to tolerate surgical procedures, (2) cognitively impaired—incapable of learning the protocol, (3) unable to adequately communicate to obtain proper consent and follow instructions, or (4) a lesion or atrophy in the intended implant area.21 A history suggesting poor wound healing, chronic infections, or cancer should be exclusion criteria, as these will impact ability to evaluate and use any type of interface. Furthermore, patients who are angry about their injury or emotionally unstable or depressed should be avoided. Acute injuries are to be avoided until it is clear that spontaneous recovery will not occur. On the other hand, it is probably reasonable to proceed with the implantation as soon as a probable permanent deficit is identified because the potential for plasticity is needed. However, studies from stroke and SCI patients demonstrate the potential for plasticity changes even months or years following injury so timing may be less critical than originally thought.22–25 The close interaction between neural interface devices and neuroplasticity via motor imagery training suggests that neural interface devices may improve or retain functional recovery.
Lead Configuration
There are basically three types of electrodes considered useful for recording cortical or subcortical activity: nonpenetrating, penetrating, and bioelectrode (Fig. 120-5). The characteristics of the electrode will determine which localizations are possible and the electronics follow. There are nonpenetrating surface recording electrodes, either from scalp EEG26–29 or from pial surface electrocorticogram (ECoG).30–32 The EEG activity is in the microvolt range and occasionally can be difficult to separate from electrical noise in the environment. The advantages are the ease of use, portability, temporal resolution, low setup cost, and low risks compared to invasive interfaces. The disadvantages are poor spatial resolution (field potentials recording a 3- to 5-cm radius), poor signal-to-noise separation, susceptibility to electromyographic noise, the extensive training required, and the low volume of output. Communication devices based on scalp EEG are commercially available. Recent applications have found mu and beta waves to be the most effective for BCI as they decrease activity during preparation and movement and thus reflect motor output pathways even for imagined movements.33 Patients with ALS or SCI have learned to use scalp EEG signals to communicate and operate simple orthoses.26,27 Steady-state, visual-evoked potentials can be used as an electrophysiologic correlate of visual spatial attention that may be harnessed to achieve control in BCI.34 Self-regulation of slow cortical potentials has been tested for BCI communication but is compromised by high intersubject variability.35
FIGURE 120-5 This schematic shows essential components of BCI system. The essential elements to practical functioning of a BCI platform follow.1 Signal acquisition: The BCI system’s recorded brain signal input (i.e., EEG, ECoG, single unit action potentials). This signal is then digitized for analysis.2 Signal processing, conversion of raw information into a useful device command: This involves both feature extraction, determination of a meaningful change in signal, and feature translation, the conversion of that signal alteration to a device command.3 Device output, overt command, or control functions administered by the BCI system. These outputs can range from simple forms of basic word processing and communication to higher levels of control such as driving a wheelchair or controlling a prosthetic limb. As a new output channel, the user must have feedback on his or her overt device output to improve performance of how to alter the electrophysiological signal. BCI, brain–computer interface.
(From Leuthardt EC, Schalk C, Moran D, et al. The emerging world of motor neuroprosthetics: a neurosurgical perspective. Neurosurgery. 2006;59:1-14, with permission.)
The ECoG is called a partially invasive BCI because the leads are implanted inside the skull but rest outside the brain rather than within the gray matter. The signal is far more robust (magnitude of five times) than scalp EEG where the cranium deflects and deforms signals. Signals can be acquired using either a subdural or epidural electrode placement. ECoG is a very promising BCI modality because it has higher spatial resolution (0.5–1 cm), better signal-to-noise ratio, wider frequency range, better long-term stability, and fewer training requirements than EEG BCI. There is a very low risk of cortical or leptomeningeal scar-tissue formation that will obscure the signal.36,37 The basic technique for electrode placement is well established in the treatment of epilepsy, and thus transfer to BCI has lower technical difficulty and lower clinical risk than invasive single-neuron recording. A disadvantage is that these are still relatively large regions that may or may not be able to be specifically and separately controlled for three-dimensional movements. Because the electrodes are on the surface, they can move and lose specific registration. Another disadvantage is the apparent low rate of information transfer. Signaling still requires detection of frequency changes, which require averaging that slows detection and data transfer. Currently, the study patients had severe epilepsy that required temporarily implanted invasive monitoring for localization prior to resection of an epileptogenic focus with rare exception.38 There are specifically designed cortical recording electrodes in development to provide better signals for BCI.
ECoG BCI was first trialed in humans in 2004 by Leuthardt and colleagues. The use of ECoG activity in closed-loop real-time control of a cursor to increase flexibility and potential utility of such recordings has been demonstrated, in association with motor or speech imagery.30 Recently, implanted 16-microwire arrays (1-mm electrode spacing) placed over the primary motor cortex while the patient performed simple contra- and ipsi-lateral wrist movements demonstrated that small regions of primary motor cortex (<5 mm) carry sufficient information to separate multiple facets of motor movements.31 Others have indicated that ECoG from other cortical regions involved in higher cognitive functions may serve as a readily self-controllable input for BCI.32 This research indicates that control is rapid, requires minimal training, and may be an ideal tradeoff with regard to signal fidelity and level of invasiveness. This suggests that a high level of control with minimal training requirements may be possible with ECoG BCI for people with motor disabilities.
The other types of electrode are penetrating. Implanted in the gray matter, these invasive devices produce the highest quality signals for BCI but are prone to scar tissue. This is the most popular electrode of BCI as the time scale shifts from 1 to 2 seconds to 200 to 400 milliseconds. These are implanted either in the cortical or subcortical areas. They may be individual or multiple. They may record field potentials (1 mm) or individual units (200–400 microns). High-amplitude local-field potential (LFP) recordings have the advantages of not requiring specific units to be identified, and theoretically can be maintained indefinitely by simply recording activity over a discrete area. In the motor cortex, these represent neural activity from widespread neurons that are temporally coupled and related to motor planning and preparatory activity rather than precise motor encoding.14 Elsewhere, LFP in parietal cortex may be useful to recognize intention and planning.39,40 Intracortical LFP signals were able to be used in a crude manner to control digital movements of a cyber hand.41 This system requires threshold data and there is only a very short detection delay. Extracranially obtained LFP can be used, but appear to be less useful due to limited information content and loss of signal. The disadvantages of LFP recording are that the field must respond to conditioning, which is in some ways more difficult than individual units and control of multiple degrees of freedom has not been demonstrated. These appear quite capable of providing binary or switching function in real time, but lack the robustness of proportional responses for fine prosthesis movements.
The most commonly used invasive electrodes are multiple prongs (Fig. 120-4A), such as the Utah or Michigan electrode arrays.42,43 Various versions of these arrays all suffer from a number of basic problems that have recently been reviewed.44,45 Over time, units are lost in what has been called the “fork in the Jell-O problem.” The brain is in constant motion and the tines can injure the tissue resulting in gliosis and damage to blood vessels and neurons with an approximately 100-micron radius.46 Designing the probe as a platform removes the handle of the fork but the connections can still add torque to the platform in terms of micromovements in response to Valsalva maneuvers or any brain movement such as from minor trauma. The presence of local electrical inhomogeneities (encapsulation, edema, coatings) around the electrode shank can substantially degrade the neural recordings. This has been frequently seen in monkey studies and clinical studies. There is then difficulty determining whether the same unit can be continuously recorded over time, and therefore, currently reregistration of the units is performed on a daily basis. With time, gliosis can surround the electrode and insulate it so that no action potential can be recorded.47 This provides a significant disadvantage for long-term programming and conditioning. An alternative is an array of microwires that can be implanted in loose bundles48 or fixed grids.49 It is unknown whether these behave differently in subcortical locations versus cortical, but it is unlikely that they would. The multitude of leads presents multiple problems for transmitting all the signals.
A new generation of electrodes should be based on the following known challenges:
