1. fundus photography (including so-called red-free photography): image intensities represent the amount of reflected light of a specific waveband

2. color fundus photography: image intensities represent the amount of reflected red (R), green (G), and blue (B) wavebands, as determined by the spectral sensitivity of the sensor

3. stereo fundus photography: image intensities represent the amount of reflected light from two or more different view angles for depth resolution

4. SLO: image intensities represent the amount of reflected single-wavelength laser light obtained in a time sequence

5. adaptive optics SLO: image intensities represent the amount of reflected laser light optically corrected by modeling the aberrations in its wavefront

6. fluorescein angiography and indocyanine angiography: image intensities represent the amounts of emitted photons from the fluorescein or indocyanine green fluorophore that was injected into the subject’s circulation.

Time domain OCT

With time domain OCT, the reference mirror is moved mechanically to different positions, resulting in different flight time delays for the reference arm light. Because the speed at which the mirror can be moved is mechanically limited, only thousands of A-scans can be obtained per second. The envelope of the interferogram determines the intensity at each depth.¹³ The ability to image the retina two-dimensionally and three-dimensionally depends on the number of A-scans that can be acquired over time. Because of motion artifacts such as saccades, safety requirements limiting the amount of light that can be projected on to the retina, and patient comfort, 1–3 seconds per image or volume is essentially the limit of acceptance. Thus, the commercially available time domain OCT, which allowed collecting of up to 400 A-scans per second, has not yet been suitable for 3D imaging.

Frequency domain OCT

In frequency domain OCT, broadband interference is acquired with spectrally separated detectors, either by encoding the optical frequency in time with a spectrally scanning source or with a dispersive detector, like a grating and a linear detector array. The depth scan can be immediately calculated by Fourier transform from the acquired spectra, without movement of the reference arm. This feature improves imaging speed dramatically, while the reduced losses during a single scan improve the signal to noise proportional to the number of detection elements. The parallel detection at multiple-wavelength ranges limits the scanning range, while the full spectral bandwidth sets the axial resolution.

Portable, cost-effective fundus imaging

For early detection and screening, the optimal place for positioning fundus cameras is at the point of care: primary care clinics, public venues (e.g., drug stores, shopping malls). Though the transition from film-based to digital fundus imaging has revolutionized the art of fundus imaging and made telemedicine applications feasible, the current cameras are still too bulky, expensive, and may be difficult to use for untrained staff in places lacking ophthalmic imaging expertise. Several groups are attempting to create more cost-effective and easier-to-use handheld fundus cameras, employing a variety of technical approaches.23,24

Functional imaging

For the patient as well as for the clinician, the outcome of disease management is mainly concerned with the resulting organ function, not its structure. In ophthalmology, current functional testing is mostly subjective and patient-dependent, such as assessing visual acuity and utilizing perimetry, which are all psychophysical metrics. Among more recently developed “objective” techniques, oxymetry is a hyperspectral imaging technique in which multispectral reflectance is used to estimate the concentration of oxygenated and deoxygenated hemoglobin in the retinal tissue.²⁵ The principle allowing the detection of such differences is simple: deoxygenated hemoglobin reflects longer wavelengths better than does oxygenated hemoglobin. Nevertheless, measuring absolute oxygenation levels with reflected light is difficult because of the large variety in retinal reflection across individuals and the variability caused by the imaging process. The retinal reflectance can be modeled by a system of equations, and this system is typically underconstrained if this variability is not accounted for adequately. Increasingly sophisticated reflectance models have been developed to correct for the underlying variability, with some reported success.²⁶ Near-infrared fundus reflectance in response to visual stimuli is another way to determine the retinal function in vivo and has been successful in cats. Initial progress has also been demonstrated in humans.²⁷

Adaptive optics

The optical properties of the normal eye result in a point spread function width approximately the size of a photoreceptor. It is therefore impossible to image individual cells or cell structure using standard fundus cameras because of aberrations in the human optical system. Adaptive optics uses mechanically activated mirrors to correct the wavefront aberrations of the light reflected from the retina, and thus has allowed individual photoreceptors to be imaged in vivo.²⁸ Imaging other cells, especially the clinically highly important ganglion cells, has thus far been unsuccessful in humans.

Longer-wavelength OCT imaging

3D OCT imaging is now the clinical standard of care for several eye diseases. The wavelengths around 840 µm used in currently available devices are optimized for imaging of the retina. Deeper structures, such as the choroidal vessels, which are important for AMD and other choroidal diseases, and the lamina cribrosa, relevant for glaucomatous damage, are not as well depicted. Because longer wavelengths penetrate deeper into the tissue, a major research effort has been undertaken to develop low-coherence swept source lasers with center wavelengths of 1000–1300 µm. Prototypes of these devices are already able to resolve detail in the choroid and lamina cribrosa.²⁹

Early detection of diabetic retinopathy

Early detection of DR via population screening associated with timely treatment has been shown to prevent visual loss and blindness in patients with retinal complications of diabetes.31,32 Almost 50% of people with diabetes in the USA currently do not undergo any form of regular documented dilated eye exam, in spite of guidelines published by the American Diabetes Association, the American Academy of Ophthalmology, and the American Optometric Association.³³ In the UK, a smaller proportion or approximately 20% of diabetics are not regularly evaluated, as a result of an aggressive effort to increase screening for people with diabetes. Blindness and visual loss can be prevented through early detection and timely management. There is widespread consensus that regular early detection of DR via screening is necessary and cost-effective in patients with diabetes.^34–37 Remote digital imaging and ophthalmologist expert reading have been shown to be comparable or superior to an office visit for assessing DR and have been suggested as an approach to make the dilated eye exam available to unserved and underserved populations that do not receive regular exams by eye care providers.^38,39 If all of these underserved populations were to be provided with digital imaging, the annual number of retinal images requiring evaluation would exceed 32 million in the USA alone (approximately 40% of people with diabetes with at least two photographs per eye).^39,40 In the next decade, projections for the USA are that the average age will increase, the number of people with diabetes in each age category will increase, and there will be an undersupply of qualified eye care providers, at least in the near term. Several European countries have successfully instigated in their healthcare systems early detection programs for DR using digital photography with reading of the images by human experts. In the UK, 1.7 million people with diabetes were screened for DR in 2007–2008. In the Netherlands, over 30 000 people with diabetes were screened since 2001 in the same period, through an early-detection project called EyeCheck.⁴¹ The US Department of Veterans Affairs has deployed a successful photo screening program through which more than 120 000 veterans were screened in 2008. While the remote imaging followed by human expert diagnosis approach was shown to be successful for a limited number of participants, the current challenge is to make the early detection more accessible by reducing the cost and staffing levels required, while maintaining or improving DR detection performance. This challenge can be met by utilizing computer-assisted or fully automated methods for detection of DR in retinal images.^42–44

Early detection of systemic disease from fundus photography

In addition to detecting DR and age-related macular degeneration, it also deserves mention that fundus photography allows certain cardiovascular risk factors to be determined. Such metrics are primarily based on measurement of retinal vessel properties, such as the arterial to venous diameter ratio, and indicate the risk for stroke, hypertension, or myocardial infarct.45,46

Image-guided therapy for retinal diseases with 3D OCT

With the introduction of 3D OCT imaging, the wealth of new information about retinal morphology has enabled its usage for close monitoring of retinal disease status and guidance of retinal therapies. The most obvious example of successful image-guided management in ophthalmology is its use in diabetic macular edema. Currently, OCT imaging is widely used to determine the extent and amount of retinal thickening. More detailed analyses of retinal layer morphology and texture from OCT will allow direct image-based treatment to be guided by computer-supported or automated quantitative analysis. This can be subsequently optimized, allowing a personalized approach to retinal disease treatment to become a reality.

Another highly relevant example of a disease that will benefit from image-guided therapy is exudative age-related macular degeneration. With the advent of the anti-vascular endothelial growth factor (VEGF) agents ranibizumab and bevacizumab, it has become clear that outer retinal and subretinal fluid is the main indicator of a need for anti-VEGF retreatment.47–51 Several studies are under way to determine whether OCT-based quantification of fluid parameters and affected retinal tissue can help improve the management of patients with anti-VEGF agents.

The retinal image

Retinal image analysis

Image analysis is a process by which meaningful information or measurements can be extracted from digital images, typically by computer algorithms. In ophthalmology, image analysis is primarily used to extract clinically relevant measurements from images of the eye, but also to estimate retinal biomarkers, most commonly from fundus color images and from OCT images. The purpose of this section is to familiarize the reader with the main concepts used in the ophthalmic image analysis literature. Image analysis is best understood as a process consisting of a combination of steps. Not all steps are performed in all image analysis algorithms, and some steps may be explicit as multiple steps in one algorithm and form a combined step in another, different algorithm, but the steps described below are typical.

Measuring performance of image analysis algorithms

Crucial for the acceptance of image analysis algorithms are evaluations of its performance. Most often performance is compared to human experts, though this raises its own set of issues, as explained below. The agreement between an automatic system and an expert reader may be affected by many influences – system performance may become impaired due to the algorithmic limitations, the imaging protocol, properties of the camera used to acquire the fundus images, and a number of other causes. For example, an imaging protocol that does not allow small lesions to be depicted and thus detected will lead to an artificially overestimated system performance if such small lesions might have been detected with an improved camera or better imaging protocol. Such a system then appears to be performing better than it truly is if human experts and the algorithm both overlook true lesions.

Detection of retinal vessels

Automated segmentation of retinal vessels has been highly successful in the detection of large and medium vessels57–59 (Fig. 6.3). Because retinal vessel diameter and especially the relative diameters of arteries and veins are known to signal the risk of systemic disease, including stroke, accurate determination of retinal vessel diameters, as well as the ability to differentiate veins from arteries, has become more important. Several semiautomated and automated approaches to determining vessel diameter have now been published.^60–62 Other active areas of research include separation of arteries and veins, detection of small vessels with diameters of less than a pixel, and analysis of complete vessel trees using graphs.

Vessel detection approaches can be divided into region-based and edge-based approaches. Region-based segmentation methods label each pixel as either inside or outside a blood vessel. Niemeijer et al. proposed a pixel-based retinal vessel detection method using a gaussian derivative filter bank and k-nearest-neighbor classification.57,59 Staal et al.⁵⁹ proposed a pixel feature-based method that additionally analyzed the vessels as elongated structures. Edge-based methods can be further classified into two categories: window-based methods and tracking-based methods. Window-based methods estimate a match at each pixel against the pixel’s surrounding window. The tracking approach exploits local image properties to trace the vessels from an initial point. A tracking approach can better maintain the connectivity of vessel structure. Lalonde et al.⁶³ proposed a vessel-tracking method by following an edge line while monitoring the connectivity of its twin border on a vessel map computed using a Canny edge operator. Breaks in the connectivity will trigger the creation of seeds that serve as extra starting points for further tracking. Gang et al. proposed a retinal vessel detection using a second-order gaussian filter with adaptive filter width and adaptive threshold.⁶⁴

Detection of fovea and optic disc

Location of the optic disc and fovea benefits retinal image analysis (Fig. 6.4). It is often necessary to mask out the normal anatomy before finding abnormal structures. For instance, the optic disc might be mistaken for a bright lesion if not detected. Secondly, the distribution of the abnormalities is not uniform on fundus photographs. Specific abnormalities occur more often in specific areas on the retina. Most optic disc detection methods are based on the fact that the optic disc is the convergence point of blood vessels and it is normally the brightest structure on a fundus image. Most fovea detection methods depend partially on the result of optic disc detection.

Hoover et al. proposed a method for optic disc detection based on the combination of vessel structure and pixel brightness.⁶⁵ If a strong vessel convergence point is found in the image, it is regarded as the optic disc. Otherwise the brightest region is detected. Foracchia et al. proposed an optic disc detection method based on vessel directions.⁶⁶ A parabolic model of the main vascular arches is established and the model parameters are the directions associated with different locations on the parabolic model. The point with a minimum sum of square error is reported as the optic disc location. Lowell et al., by matching an optic disc model using the Pearson correlation, determined an initial optic disc location, and then traced the optic disc boundary using a deformable contour model.⁶⁷

Most fovea detection methods use the fact that the fovea is a dark region in the image and that it normally lies in a fixed orientation and location relative to the optic disc and the main vascular arch. In a study by Fleming et al., approximate locations of the optic disc and fovea are obtained using the elliptical form of the main vascular arch.⁶⁸ Then, the locations are refined based on the circular edge of the optic disc and the local darkness at the fovea. Li and Chutatape also proposed a method to select the brightest 1% pixels in a gray-level image.⁶⁹ The pixels are clustered and principal component analysis based on a trained system is applied to extract a single point as the estimated location of the optic disc. A fovea candidate region is then selected based on the optic disc location and the main vascular arch shape. Within the candidate region, the centroid of the cluster with the lowest mean intensity and pixel number greater than one-sixth disc area is regarded as the foveal location. In a paper by Sinthanayothin et al., the optic disc was located as the area with the highest variation in intensity of adjacent pixels, while the fovea was extracted using intensity information and a relative distance to the optic disc.⁷⁰ Tobin et al. proposed a method to detect the optic disc based on blood vessel features, such as density, average thickness, and orientation.⁷¹ Then the fovea location was determined based on the location of the optic disc and a geometry model of the main blood vessel. Niemeijer et al. proposed a method to localize automatically both the optic disc and fovea in 2006.^72,73 For the optic disc detection, a set of features are extracted from the color fundus image. A k-nearest-neighbor classification is used to give a soft label to each pixel on the test image. The probability image is blurred and the pixel with the highest probability is detected as optic disc. Relative position information between the optic disc and the fovea is used to limit the search of fovea into a certain region. For each possible location of the optic disc, a possible location of the fovea is given. The possible locations for the fovea are stored in a separate image and the highest-probability location is detected as the fovea location.

Detection of retinal lesions

In this section we will primarily focus on detection of lesions in DR. DR has the longest history as a research subject in retinal image analysis. Figure 6.4 shows examples of a fundus photograph with the typical lesions automatically detected. After preprocessing, most approaches detect candidate lesions, after which a mathematical morphology template is utilized to segment and characterize the candidates (Fig. 6.5). This approach or a modification thereof is in use in many algorithms for detecting DR and age-related macular degeneration.⁷⁴ Additional enhancements include the contributions of Spencer et al.⁷⁵ and Frame et al.⁷⁶ They added additional preprocessing steps, such as shade correction and matched filter postprocessing, to this basic framework to improve algorithm performance. Algorithms of this kind function by detecting candidate microaneurysms of various shapes, based on their response to specific image filters. A supervised classifier is typically developed to separate valid microaneurysms from spurious or false responses. However, these algorithms were originally developed to detect the high-contrast signatures of microaneurysms in fluorescein angiogram images. An important development was the addition of a more sophisticated filter, a modified version of the top-hat filter, so called because of its cross-section, to red-free fundus photographs rather than angiogram images, as was first described by Hipwell et al.⁷⁷ They tested their algorithm on a large set of >3500 images and found a sensitivity/specificity operating point of 0.85/0.76. Once this filter-based approach had been established, development accelerated. The next step was broadening the candidate detection step, originally developed by Baudoin¹⁶ to detect candidate pixels, to a multifilter filter-bank approach.^57,78 The responses of the filters are used to identify pixel candidates using a classification scheme. Mathematical morphology and additional classification steps are applied to these candidates to decide whether they indeed represent microaneurysms and hemorrhages (Fig. 6.6). A similar approach was also successful in detecting other types of DR lesions, including exudates and cotton-wool spots, as well as drusen in AMD.⁷⁹

Small red retinal lesions, namely microaneurysms and small retinal hemorrhages, are typical for multiple retinal disorders, including DR, hypertensive retinopathy, venous occlusive disease, and other less common retinal disorders such as idiopathic juxtafoveal telangiectasia. The primary importance of small red lesions is that they are the leading indicators of DR. Because they are difficult to differentiate for clinicians on standard fundus images from nonmydriatic cameras, hemorrhages and microaneurysms are usually detected together and associated with a single combined label. Historically, red lesion detection algorithms focused on detection of normal anatomical objects, especially the vessels, because they can locally mimic red lesions. Subsequently, a combination of one or more filtering operations combined with mathematical morphology is employed to detect red lesion suspects. In some cases, suspect red lesions are further classified into individual lesion types and refined algorithms are capable of detecting specific retinal structures and abnormalities.

Initially, red lesions were detected in fluorescein angiograms because their contrast against the background is much higher than that of microaneurysms in color fundus photography images.75,76,80 Hemorrhages mask out fluorescence and present as dark spots in the angiograms. These methods employed a mathematical morphology technique that eliminated the vasculature from a fundus image but left possible microaneurysm candidates untouched, as first described in 1984.¹⁶ Later, this method was extended to high-resolution red-free fundus photographs by Hipwell et al.⁷⁷ Instead of using morphology operations, a neural network was used, as demonstrated by Gardner et al.⁸¹ In their work, images are divided into 20 × 20 pixel grids and the grids are individually classified. Sinthanayothin et al. used a detection step to find blood-like regions and to segment both vessels and red lesions in a fundus image.⁸² A neural network was used to detect the vessels exclusively, and the remaining objects were labeled as microaneurysms. Niemeijer et al. presented a hybrid scheme that used a supervised pixel classification-based method to detect and segment the microaneurysm candidates in color fundus photographs.⁷⁸ This method allowed for the detection of larger red lesions (i.e., hemorrhages) in addition to the microaneurysms using the same system. A large set of additional features, including color, was added to those that had been previously described.^76,80 Using the features in a supervised classifier distinguished between real and spurious candidate lesions. These algorithms can usually distinguish between overlapping microaneurysms because they give multiple candidate responses.

Other recent algorithms only detect microaneurysms and forgo a phase of detecting normal retinal structures like the optic disc, fovea, and retinal vessels, which can act as confounders for abnormal lesions. Instead, the recent approaches find the microaneurysms directly using template matching in wavelet subbands.⁸³ In this approach, the optimal adapted wavelet transform is found using a lifting scheme framework. By applying a threshold on the matching result of the wavelet template, the microaneurysms are labeled. This approach has meanwhile been extended explicitly to account for false negatives and false positives.⁴² Because it avoids detection of the normal structures, such algorithms can be very fast, on the order of less than a second per image.

Bright lesions, defined as lesions brighter than the retinal background, can be found in the presence of retinal and systemic disease. Some examples of such bright lesions of clinical interest include drusen, cotton-wool spots, and lipoprotein exudates. To complicate the analysis, flash artifacts can be present as false positives for bright lesions. If the lipoprotein exudates only appear in combination with red lesions, they would only be useful for grading DR. The exudates can, however, in some cases appear as isolated signs of DR in the absence of any other lesion. Several computer-based systems to detect exudates have been proposed (Fig. 6.7).^{74,79,81,82,84}

Because the different types of bright lesion have different diagnostic importance, algorithms should be capable not only of detecting bright lesions, but also of differentiating among the bright lesion types. One example algorithm capable of detection and differentiation of bright lesions was reported by Niemeijer et al. in 2007.⁷⁹ This algorithm is based on an earlier red lesion algorithm presented by Hipwell et al. in 2000⁷⁷ and includes the following traditional steps, which are illustrated in Fig. 6.6:

Vessel analysis

Vessel measures, such as the average width of arterioles and venules, the ratio of arteriolar to venular widths, and the branching ratio, have been established to be predictive of systemic diseases, especially hypertension, and also have potential value in degenerative retinal diseases such as retinitis pigmentosa. The methods in the section on detection of retinal vessels locate the vessels, but cannot determine vessel width. Additional techniques are needed to measure the vessel width accurately. Al-Diri et al. proposed an algorithm for segmentation and measurement of retinal blood vessels by growing a “ribbon of twins” active contour model. Their approach uses an extraction of segment profiles algorithm, which uses two pairs of contours to capture each vessel edge.⁸⁵ The half-height full-width algorithm defines the width as the distance between the points on the intensity curve at which the function reaches half its maximum value to either side of the estimated center point.^86–88 The Gregson algorithm fits a rectangle to the profile, setting the width so that the area under the rectangle is equal to the area under the profile.³³ Xu et al. recently published a method based on graph search showing less variability than human experts (Fig. 6.8).⁸⁹

A fully automated method from the Abramoff group to measure the arteriovenous ratio in disc center retinal images was recently published.⁹⁰ This method detects the location of the optic disc, determines an appropriate region of interest, classifies vessels as arteries or veins, estimates vessel widths, and calculates the arteriovenous ratio. The system eliminates all vessels outside the arteriovenous ratio measurement region of interest. A skeletonization operation is then applied to the remaining vessels after which vessel crossings and bifurcation points are removed, leaving a set of vessel segments consisting of only vessel centerline pixels. Features are extracted from each centerline pixel in order to assign these a soft label indicating the likelihood that the pixel is part of a vein. As all centerline pixels in a connected vessel segment should be the same type, the median soft label is assigned to each centerline pixel in the segment. Next, artery vein pairs are matched using an iterative algorithm, and finally, the widths of the vessels are used to calculate the average arteriovenous ratio.

Retinal atlas

The retina has a relatively small number of key anatomic structures (landmarks) visible using planar fundus camera imaging. Additionally, the expected shape, size, and color variations across a population are expected to be high. While there have been a few reports on estimating retinal anatomic structure using a single retinal image,⁷¹ we are not aware of any published work demonstrating the construction of a statistical retinal atlas using data from a large number of subjects. The choice of atlas landmarks in retinal images may vary depending on the view of interest. Regardless, the atlas should represent most retinal image properties in a concise and intuitive way. Three landmarks can be used as the retinal atlas key features: the optic disc center, the fovea, and the main vessel arch defined as the location of the largest vein/artery pairs. The disc and fovea provide landmark points, while the arch is a more complicated two-part curved structure that can be represented by its central axis. The atlas coordinate system then defines an intrinsic, anatomically meaningful framework within which anatomic size, shape, color, and other characteristics can be objectively measured and compared. Choosing either the disc center or fovea alone to define the atlas coordinate system would allow each image from the population to be translated so pinpoint alignment can be achieved. Choosing both disc and fovea allows corrections for translation, scale, and rotational differences across the population. However, nonlinear shape variations across the population would not be considered – this can be accomplished when the vascular arch information is utilized. The end of the arches can be defined as the first major bifurcations of the arch branches. The arch shape and orientation vary from individual to individual and influence the structure of the remaining vessel network. Establishing an atlas coordinate system that incorporates the disc, fovea, and arches allows for translation, rotation, scaling, and nonlinear shape variations to be accommodated across a population.

An isotropic coordinate system is a system in which the size of each imaged element is the same in all three dimensions. This is desirable for a retinal atlas so images can refer to the atlas independent of spatial pixel location by a linear one-to-one mapping. The radial distortion correction (RADIC) model attempts to register images in a distortion-free coordinate system using a planar-to-spherical transformation, so the registered image is isotropic under a perfect registration, and places the registered image in an isotropic coordinate system (Fig. 6.9).⁹¹ An isotropic atlas makes it independent of spatial location to map correspondences between the atlas and test image. The intensities in overlapping area are determined by a distance-weighted blending scheme.⁹²

Retinal images in clinical practice are acquired under diverse fundus camera settings subjected to saccadic eye movement, and with variable properties, including focal center, zoom, and tilt. Thus, atlas landmarks from training data need to be aligned to derive any meaningful statistical properties from the atlas. Since the projective distortion within an image is corrected in registration, the interimage variations in the registered images appear as the difference in the rigid coordinate transformation parameters of translation, scale, and rotation.

The atlas landmarks serve as the reference set so each color fundus image can be mapped to the coordinate system defined by the landmarks. As the last step of atlas generation, color fundus images are warped to the atlas coordinate system so that the arch of each image is aligned to the atlas vascular arch (Fig. 6.10).⁹³ Rigid coordinate alignment is done for each fundus image to register the disc center and the fovea. The control points are determined by sampling points from equidistant locations in radial directions from the disc center. Usually, the sampling uses smoothed trace lines utilizing third-order polynomial curve fitting to eliminate locally high tortuosity of vascular tracings which may cause large geometric distortions (Fig. 6.11).

A retinal atlas can be used as a reference to quantitatively assess the level of deviation from normality. An analyzed image can be compared with the retinal atlas directly in the atlas coordinate space. The normality can thus be defined in several ways depending on the application purpose – using local or global chromatic distribution, degree of vessel tortuosity, presence of pathological features, or presence of artifacts (Fig. 6.12). Other uses for a retinal atlas include image quality detection and disease severity assessment. Retinal atlases can also be employed in content-based image retrieval leading to abnormality detection in retinal images.⁹⁴

Performance of DR detection algorithms

Several groups have studied the performance of detection algorithms in a real-world setting. The main goal of such a system is to decide whether the patient should be evaluated by a human expert or can return for routine follow-up, based solely on automated analysis of retinal images.43,44

DR detection algorithms appear to be mature and competitive algorithms and have now reached the human intrareader variability limit.41,42,95 Additional validation studies on larger, well-defined, but more diverse populations of patients with diabetes are urgently needed; anticipating cost-effective early detection of DR in millions of people with diabetes to triage those patients who need further care at a time when they have early rather than advanced DR. Validation trials are currently under way in the USA, UK, and the Netherlands.

To drive the development of progressively better fundus image analysis methods, research groups have established publicly available, annotated image databases in various fields. Fundus imaging examples are represented by the STARE,⁹⁶ DRIVE,⁵⁷ REVIEW,⁹⁷ and MESSIDOR databases,⁹⁸ with large numbers of annotated retinal fundus images, with expert annotations for vessel segmentation, vessel width measurements, and DR detection. Image analysis competitions such as the Retinopathy Online Challenge⁹⁵ have also been initiated.

The Digital Retinal Images for Vessel Evaluation (DRIVE) database was established to enable comparative studies on segmentation of retinal blood vessels in retinal fundus images. It contains 40 fundus images from subjects with diabetes, both with and without retinopathy, as well as retinal vessel segmentations performed by two human observers. Starting in 2005, researchers have been invited to test their algorithms on this database and share their results with other researchers through the DRIVE website.⁸⁹ At the same web location, results of various methods can be found and compared. Currently, retinal vessel segmentation research is primarily focusing on improved segmentation of small vessels, as well as on segmenting vessels in images with substantial abnormalities.

The DRIVE database was a great success, allowing comparisons of algorithms on a common data set. In retinal image analysis, it represented a substantial improvement over method evaluations on unknown data sets. However, different groups of researchers tend to use different metrics to compare the algorithm performance, making truly meaningful comparisons difficult or impossible. Additionally, even when using the same evaluation measures, implementation specifics of the performance metrics may influence final results. Consequently, until the advent of the Retinopathy Online Challenge competition in 2009, comparing the performance of retinal image analysis algorithms was difficult.⁹⁵ This competition focused on detection of microaneurysms. Twenty-six groups participated in the competition, out of which six groups submitted their results on time. The results from each of the methods in this competition are summarized in Table 6.1.

A logical next step was to provide publicly available annotated data sets for use in the context of online, standardized, asynchronous competitions. In an asynchronous competition, a subset of images is made available with annotations, while the remainder of the images are available with annotations withheld. This allows researchers to optimize their algorithm performance on the population from which the images were drawn (assuming the subset with annotated images is representative of the entire population), but they are unable to test–retest on the evaluation images, because those annotations are withheld. All results are subsequently evaluated using the same evaluation software and research groups are allowed to submit results continuously over time.

Areas of active research in fundus image analysis

Major progress has been accomplished in fundus image analysis. Current challenges, on which multiple research groups worldwide are actively working, include the following areas: differentiating arteries from veins, assessing accurate vessel diameter (particularly in vessels only a few pixels in diameter) and vessel tortuosity, vessel tree analysis including tree branching patterns, detection of irregularly shaped hemorrhages, detection of lesion distribution patterns (i.e., drusen), and segmentation of atrophy. Finally, integration of fundus image-based quantification with other metrics of disease risk, such as serum glucose level or patient history, is an area of active research with immediate clinical application.

Retinal layer analysis from 3D OCT

Detection of retinal vessels from 3D OCT

Segmenting the retinal vasculature in 3D SD-OCT volumes129,130 allows OCT-to-fundus and OCT-to-OCT image registration. The absorption of light by the blood vessel walls causes vessel silhouettes to appear below the position of vessels, which thus causes the projected vessel positions to appear dark on either a full projection image of the entire volume¹³⁰ or a projection image from a segmented layer for which the contrast between the vascular silhouettes and background is highest, as proposed by Niemeijer et al.^129,131 In particular, this approach used the layer near the retinal pigment epithelium to create the projection image. Vessels were segmented using feature detection with gaussian filter-banks and a k-NN pixel classification approach. The performance of the automated method was evaluated for both optic nerve head-centered as well as macula-centered scans. The retinal vessels were successfully identified in a set of 16 3D OCT volumes (8 optic nerve head and 8 macula-centered), with high sensitivity and specificity as determined using ROC analysis, AUC = 0.96. Xu et al. reported an approach for segmenting the projected locations of the vasculature by utilizing pixel classification of A-scans.¹³² The features used in the pixel classification were based on a projection image of the entire volume in combination with features of individual A-scans. Both of these reported prior approaches focused on segmenting the vessels in the region outside the optic disc region because of difficulties in the segmentation inside this region. The neural canal opening shares similar features with vessels, thus causing false positives.¹³³ Hu et al.¹²⁷ proposed a modified 2D pixel classification algorithm to segment the blood vessels in SD-OCT volumes centered at the optic nerve head, with a special focus on better identifying vessels near the neural canal opening. Given an initial 2D segmentation of the projected vasculature, Lee et al. presented an approach for segmenting the 3D vasculature in the volumetric scans¹³⁴ by utilizing a graph-theoretic approach (Fig. 6.15). One of the current limitations of that approach is the inability to resolve the depth information of crossing vessels properly.

Detection of retinal lesions

Calculated texture and layer-based properties can be used to detect retinal lesions as a 2D footprint¹²³ or in 3D. Out of many kinds of possible retinal lesions, symptomatic exudate-associated derangements (SEADs), a general term for fluid-related abnormalities in the retina, are of great interest in assessing the severity of choroidal neovascularization, diabetic macular edema, and other diseases. Detection of drusen, cotton-wool spots, areas of pigment epithelial atrophy, or pockets of fluid under epiretinal membranes may be attempted in a similar fashion.

The deviation of local retinal tissue from normal can be computed by determining the local deviations from the normal appearance at each location (x; y) in each layer and selecting the areas where the absolute deviation is greater than a predefined cutoff. More generally, in order to build an abnormality-specific detector, a classifier can be trained, the inputs of which may be the z-scores computed for relevant features. Comprehensive z-scores are appropriate since an abnormality may affect several layers in the neighborhood of a given location (x; y). The classifier-determined label associated with each column may be selected on relevant features by one of the many available cross-validation and/or feature selection methods,135–137 thus forming a SEADness or probabilistic abnormality map.

Registration of fundus retinal photographs

Registration of fundus photographs taken either at different regions of the retina, or of the same area of the retina but at different times, is useful to expand the effective field of view of a retinal image, determine what part of the retina is being viewed, or aid in analyzing changes over time.¹⁴⁰ We have previously discussed some other uses for fundus–fundus registration in regard to retinal atlases. To register 2D or planar fundus images, most existing registration approaches utilize identification and extraction of features derived from retinal vasculature segmented separately from the individual fundus images. The choice of a specific image registration algorithm to align retinal images into a montage depends on the image characteristics and the application. Images acquired with only a small overlap may be optimally aligned using feature-based registration approaches, while images acquired with larger overlaps may be satisfactorily aligned using intensity-based approaches. Examples of feature-based registration are global-to-local matching,¹⁴¹ hierarchical model refinement,¹⁴² and dual-bootstrap.¹⁴³ Local intensity features¹⁴⁴ are particularly useful when an insufficient number of vascular features are available. Following a step of vascular skeletonization, vascular branching points can be easily used as stable landmarks for determining image-to-image correspondence. As an example, the RADIC model¹⁴⁵ parameters are estimated during an optimization step that uses Powell’s method¹⁴⁶ and is driven by the vessel centerline distance. The approach presented by Lee et al. in 2010 reported registration accuracy of 1.72 pixels (25–30 µm, depending on resolution) when tested in 462 pairs of green channel fundus images.¹⁴⁷ The registration accuracy was assessed as the vessel line error. The method only needed two correspondence points to be reliably identified and was therefore applicable even to cases when only a very small overlap between the retinal image pairs existed. Based on the identified vascular features, the general approach can be applied to any retinal imaging modality for which a 2D vessel segmentation is available. In registering poor-quality multimodal fundus image pairs, which may not have sufficient vessel-based features available, Chen et al. proposed the detection of corner points using a Harris detector followed by use of a partial intensity invariant feature descriptor.¹⁴⁸ They reported obtaining 89.9% “acceptable” registrations (defined as registrations with a median error of 1.5 pixels and a maximum error of 10 pixels when compared with ground truth correspondences) when tested on 168 pairs of multimodal retinal images.

Registration of OCT with fundus retinal photographs

Registration of 2D fundus images with inherently 3D OCT images requires that the dimensionality of OCT be reduced to 2D via z-axis projection. Building on the ability to obtain vascular segmentation from 3D OCT projection images, the problem of fundus–OCT registration becomes virtually identical to that of fundus–fundus registration that was described in the previous section. Using the same general method, high-quality OCT–fundus registration can be achieved. Figure 6.17 presents the main steps of the registration process and shows the registration performance achieved.

Mutual registration of 3D OCT images

Temporal changes of retinal layers leading to assessment of disease progression or regression can be accessed from longitudinal OCT images. Comparison of morphology or function over time requires that the respective OCT image data sets be registered. Since OCT is a 3D imaging modality, such registration needs to be performed in 3D. For follow-up studies, image registration is a vital tool to enable more precise, quantitative comparison of disease status. Another important aspect of OCT–OCT registration is the ability to enlarge retinal coverage by registering OCT data resulting from imaging different portions of the retina. A fully 3D scale-invariant feature transform (SIFT)-based approach was introduced in 2009.¹⁴⁹ The SIFT feature extractor locates minima and maxima in the difference of gaussian scale space to identify salient feature points. Using calculated histograms of local gradient directions around each found extremum in 3D, the matching points are found by comparing the distances between feature vectors. An application of this approach to rigid registration of peripapillary (ONH-centered) and macula-centered 3D OCT scans of the same patient for which the macular and peripapillary OCT scans had only a limited overlap has been reported.¹⁴⁸ The work built on a number of analysis steps introduced earlier, including segmentation of the main retinal layers and 3D flattening of each of the two volumes to be registered. 3D SIFT feature points were subsequently determined.^150,151 Using the customary terminology for image registration, when one of the registered images is called the “source” (say the macular image) and the other the “target” (say the peripapillary image), the feature point detection is performed in both source and target images. After feature point extraction, those which are in corresponding positions in both images are identified. In a typical pair of two OCT scans, about 70 matching pairs can be found with a high level of certainty. The primary deformations that need to be resolved are translation and limited rotation, so simple rigid or affine transform is appropriate to achieve the desired image registration. The transform parameters are estimated from the identified correspondence points. Niemeijer et al. demonstrated the functionality of such an approach to OCT–OCT registration of macular and peripapillary OCT scans.¹⁴⁹ 3D registration achieved 3D accuracy of 2.0 ± 3.3 voxels, assessed as an average voxel distance error in 1572 matched locations. Qualitative evaluation of performance demonstrated the utility of this approach to clinical-quality images. Temporal registration of longitudinally acquired OCT images from the same subjects can be obtained in an identical manner.

1. Zhang, X, Saaddine, JB, Chou, CF, et al. Prevalence of diabetic retinopathy in the United States, 2005–2008. JAMA. 2010;304:649–656.

2. Flick, CS. Centenary of Babbage’s ophthalmoscope. Optician. 1947;113:246.

3. Keeler, CR. 150 years since Babbage’s ophthalmoscope. Arch Ophthalmol. 1997;115:1456–1457.

4. von Helmholtz, H. Beschreibung eines Augens-Spiegels zur Untersuchung der Netzhaut im lebenden Auge. Berlin: Foerstner; 1851.

5. Trigt, AC. Dissertatio ophthalmologica inauguralis de speculo oculi. Utrecht: Universiteit van Utrecht; 1853.

6. Gerloff, O. Über die Photographie des Augenhintergrundes. Klin Monbl Augenheilkd. 1891;29:397ff.

7. Gullstrand, A. Neue methoden der reflexlosen. Ophthalmoskopie. Berichte Dtsch Ophthalmologische Gesellschaft. 1910:36.

8. Novotny, HR, Alvis, DL. A method of photographing fluorescence in circulating blood in the human retina. Circulation. 1961;24:82–86.

9. Allen, L. Ocular fundus photography: suggestions for achieving consistently good pictures and instructions for stereoscopic photography. Am J Ophthalmol. 1964;57:13–28.

10. Webb, RH, Hughes, GW. Scanning laser ophthalmoscope. IEEE Trans Biomed Eng. 1981;28:488–492.

11. Youngquist, RC, Carr, S, Davies, DE. Optical coherence-domain reflectometry: a new optical evaluation technique. Optics Lett. 1987;12:158–160.

12. Youngquist RC, Wentworth RH, Fesler KA. Selective interferometric sensing by the use of coherence synthesis. Optics Lett 1987;12:944–6.

13. Huang, D, Swanson, EA, Lin, CP, et al. Optical coherence tomography. Science. 1991;254:1178–1181.

14. Swanson, EA, Izatt, JA, Hee, MR, et al. In vivo retinal imaging by optical coherence tomography. Optics Lett. 1993;18:1864–1866.

15. Matsui, M, Tashiro, T, Matsumoto, K, et al. [A study on automatic and quantitative diagnosis of fundus photographs. I. Detection of contour line of retinal blood vessel images on color fundus photographs (author’s transl).]. Nippon Ganka Gakkai Zasshi. 1973;77:907–918.

16. Baudoin, CE, Lay, BJ, Klein, JC. Automatic detection of microaneurysms in diabetic fluorescein angiography. Rev Epidemiol Sante Publique. 1984;32:254–261.

17. Wild, S, Roglic, G, Green, A, et al. Global prevalence of diabetes: estimates for the year 2000 and projections for 2030. Diabetes Care. 2004;27:1047–1053.

18. Germain, N, Galusca, B, Deb-Joardar, N, et al. No loss of chance of diabetic retinopathy screening by endocrinologists with a digital fundus camera. Diabetes Care. 2011;34:580–585.

19. Abramoff, MD, Garvin, M, Sonka, M. Retinal imaging and image analysis. IEEE Rev Biomed Eng. 2010:169–208.

20. Fujimoto, JG. Optical coherence tomography for ultrahigh resolution in vivo imaging. Nat Biotechnol. 2003;21:1361–1367.

21. Fercher, AF, Mengedoht, K, Werner, W. Eye-length measurement by interferometry with partially coherent light. Optics Lett. 1988;13:186–188.

22. Choma, M, Sarunic, M, Yang, C, et al. Sensitivity advantage of swept source and Fourier domain optical coherence tomography. Optics Express. 2003;11:2183–2189.

23. Abramoff, MD, Kardon, RH, Vermeer, KA, et al. A portable, patient friendly scanning laser ophthalmoscope for diabetic retinopathy imaging: exudates and hemorrhages. Invest Ophthalm Vis Sci. 2007:48.

24. Hammer, DX, Ferguson, RD, Ustun, TE, et al. Line-scanning laser ophthalmoscope. J Biomed Opt. 2006;11:041126.

25. Suansilpong, A, Rawdaree, P. Accuracy of single-field nonmydriatic digital fundus image in screening for diabetic retinopathy. J Med Assoc Thai. 2008;91:1397–1403.

26. Delori, FC, Pflibsen, KP. Spectral reflectance of the human ocular fundus. Appl Opt. 1989;28:1061–1077.

27. Abramoff, MD, Kwon, YH, Ts’o, D, et al. Visual stimulus-induced changes in human near-infrared fundus reflectance. Invest Ophthalmol Vis Sci. 2006;47:715–721.

28. Whited, JD. Accuracy and reliability of teleophthalmology for diagnosing diabetic retinopathy and macular edema: a review of the literature. Diabetes Technol Ther. 2006;8:102–111.

29. Lin, DY, Blumenkranz, MS, Brothers, R. The role of digital fundus photography in diabetic retinopathy screening. Digital Diabetic Screening Group (DDSG). Diabetes Technol Ther. 1999;1:477–487.

30. Standards and Guidelines in Telehealth for Diabetic Retinopathy: American Telemedicine Association position statement. http://www.americantelemed.org/ICOT/drpositionpaper.paper.technical.htm.

31. Bresnick, GH, Mukamel, DB, Dickinson, JC, et al. A screening approach to the surveillance of patients with diabetes for the presence of vision-threatening retinopathy. Ophthalmology. 2000;107:19–24.

32. Kinyoun, JL, Martin, DC, Fujimoto, WY, et al. Ophthalmoscopy versus fundus photographs for detecting and grading diabetic retinopathy. Invest Ophthalmol Vis Sci. 1992;33:1888–1893.

33. Preferred Practice Pattern. Diabetic Retinopathy. www.aao.org/ppp.

34. Fong, DS, Aiello, L, Gardner, TW, et al. Diabetic retinopathy. Diabetes Care. 2003;26:226–229.

35. Fong, DS, Sharza, M, Chen, W, et al. Vision loss among diabetics in a group model Health Maintenance Organization (HMO). Am J Ophthalmol. 2002;133:236–241.

36. Wilson, C, Horton, M, Cavallerano, J, et al. Addition of primary care-based retinal imaging technology to an existing eye care professional referral program increased the rate of surveillance and treatment of diabetic retinopathy. Diabetes Care. 2005;28:318–322.

37. Klonoff, DC, Schwartz, DM. An economic analysis of interventions for diabetes. Diabetes Care. 2000;23:390–404.

38. Lin, DY, Blumenkranz, MS, Brothers, RJ, et al. The sensitivity and specificity of single-field nonmydriatic monochromatic digital fundus photography with remote image interpretation for diabetic retinopathy screening: a comparison with ophthalmoscopy and standardized mydriatic color photography. Am J Ophthalmol. 2002;134:204–213.

39. Williams, GA, Scott, IU, Haller, JA, et al. Single-field fundus photography for diabetic retinopathy screening: a report by the American Academy of Ophthalmology. Ophthalmology. 2004;111:1055–1062.

40. Lawrence, MG. The accuracy of digital-video retinal imaging to screen for diabetic retinopathy: an analysis of two digital-video retinal imaging systems using standard stereoscopic seven-field photography and dilated clinical examination as reference standards. Trans Am Ophthalmol Soc. 2004;102:321–340.

41. Abramoff, MD, Suttorp-Schulten, MS. Web-based screening for diabetic retinopathy in a primary care population: the EyeCheck project. Telemed J E Health. 2005;11:668–674.

42. Abramoff, MD, Reinhardt, JM, Russell, SR, et al. Automated early detection of diabetic retinopathy. Ophthalmology. 2010;117:1147–1154.

43. Abramoff, MD, Niemeijer, M, Suttorp-Schulten, MS, et al. Evaluation of a system for automatic detection of diabetic retinopathy from color fundus photographs in a large population of patients with diabetes. Diabetes Care. 2008;31:193–198.

44. Philip, S, Fleming, AD, Goatman, KA, et al. The efficacy of automated “disease/no disease” grading for diabetic retinopathy in a systematic screening programme. Br J Ophthalmol. 2007;91:1512–1517.

45. Cheung, N, Rogers, S, Couper, DJ, et al. Is diabetic retinopathy an independent risk factor for ischemic stroke? Stroke. 2007;38:398–401.

46. Photocoagulation treatment of proliferative diabetic retinopathy: the second report of diabetic retinopathy study findings. Ophthalmology. 1978;85:82–106.

47. Rosenfeld, PJ, Moshfeghi, AA, Puliafito, CA. Optical coherence tomography findings after an intravitreal injection of bevacizumab (Avastin) for neovascular age-related macular degeneration. Ophthalm Surg Lasers Imaging. 2005;36:331–335.

48. Heier, JS, Antoszyk, AN, Pavan, PR, et al. Ranibizumab for treatment of neovascular age-related macular degeneration: a phase I/II multicenter, controlled, multidose study. Ophthalmology. 2006;113:633.e1–633.e4.

49. Preliminary report on effects of photocoagulation therapy. The Diabetic Retinopathy Study Research Group. Am J Ophthalmol. 1976;81:383–396.

50. Grading diabetic retinopathy from stereoscopic color fundus photographs – an extension of the modified Airlie House classification. ETDRS report number 10. Early Treatment Diabetic Retinopathy Study Research Group. Ophthalmology. 1991;98:786–806.

51. Harvey, JN, Craney, L, Nagendran, S, et al. Towards comprehensive population-based screening for diabetic retinopathy: operation of the North Wales diabetic retinopathy screening programme using a central patient register and various screening methods. J Med Screen. 2006;13:87–92.

52. Sonka, M, Fitzpatrick, JM. Handbook of medical imaging, vol. 2. Medical image processing and analysis. Wellingham, WA: International Society for Optical Engineering Press; 2000.

53. Supplement 91: Ophthalmic Photography Image SOP Classes. ftp://medical.nema.org/medical/dicom/final/sup91_ft2.pdf.

54. DICOM. Supplement 110: Ophthalmic Tomography Image Storage SOP Class. ftp://medical.nema.org/medical/dicom/final/sup110_ft4.pdf.

55. Abramoff, MD, Alward, WL, Greenlee, EC, et al. Automated segmentation of the optic nerve head from stereo color photographs using physiologically plausible feature detectors. Invest Ophthalm Vis Sci. 2007;48:1665–1673.

56. Tobin, KW, Abramoff, MD, Chaum, E, et al. Using a patient image archive to diagnose retinopathy. Conf Proc IEEE Eng Med Biol Soc. 2008;2008:5441–5444.

57. Niemeijer, M, Staal, JS, van Ginneken, B, et al. Comparative study of retinal vessel segmentation on a new publicly available database. Proc SPIE. 2004:5370–5379.

58. Soares, JV, Leandro, JJ, Cesar Junior, RM, et al. Retinal vessel segmentation using the 2-D Gabor wavelet and supervised classification. IEEE Trans Med Imaging. 2006;25:1214–1222.

59. Staal, J, Abramoff, MD, Niemeijer, M, et al. Ridge-based vessel segmentation in color images of the retina. IEEE Trans Med Imaging. 2004;23:501–509.

60. Hubbard, LD, Brothers, RJ, King, WN, et al. Methods for evaluation of retinal microvascular abnormalities associated with hypertension/sclerosis in the Atherosclerosis Risk in Communities Study. Ophthalmology. 1999;106:2269–2280.

61. Owens, DR, Gibbins, RL, Lewis, PA, et al. Screening for diabetic retinopathy by general practitioners: ophthalmoscopy or retinal photography as 35 mm colour transparencies? Diabet Med. 1998;15:170–175.

62. Niemeijer, M, Xu, X, Dumitrescu, AV, et al. Automated measurement of the arteriolar-to-venular width ratio in digital color fundus photographs. IEEE Trans Med Imaging. 2011;30:1941–1950.

63. Lalonde, M, Gagnon, L, Boucher, M-C. Non-recursive paired tracking for vessel extraction from retinal images. Proc Conf Vis Interface. 2000:61–68.

64. Gang, L, Chutatape, O, Krishnan, SM. Detection and measurement of retinal vessels in fundus images using amplitude modified second-order Gaussian filter. IEEE Trans BioMed Eng. 2002;49:168–172.

65. Hoover, A, Goldbaum, M. Locating the optic nerve in a retinal image using the fuzzy convergence of the blood vessels. IEEE Trans Med Imaging. 2003;22:951–958.

66. Foracchia, M, Grisan, E, Ruggeri, A. Detection of optic disc in retinal images by means of a geometrical model of vessel structure. IEEE Trans Med Imaging. 2004;23:1189–1195.

67. Lowell, J, Hunter, A, Steel, D, et al. Optic nerve head segmentation. IEEE Trans Med Imaging. 2004;23:256–264.

68. Fleming, AD, Goatman, KA, Philip, S, et al. Automatic detection of retinal anatomy to assist diabetic retinopathy screening. Phys Med Biol. 2007;52:331–345.

69. Li, H, Chutatape, O. Automated feature extraction in color retinal images by a model based approach. IEEE Trans Biomed Eng. 2004;51:246–254.

70. Sinthanayothin, C, Boyce, JF, Cook, HL, et al. Automated localisation of the optic disc, fovea, and retinal blood vessels from digital colour fundus images. Br J Ophthalmol. 1999;83:902–910.

71. Tobin, KW, Chaum, E, Govindasamy, VP, et al. Detection of anatomic structures in human retinal imagery. IEEE Trans Med Imaging. 2007;26:1729–1739.

72. Abramoff, MD, Niemeijer, M. The automatic detection of the optic disc location in retinal images using optic disc location regression. Conf Proc IEEE Eng Med Biol Soc. 2006;1:4432–4435.

73. Niemeijer, M, Abramoff, MD, van Ginneken, B. Segmentation of the optic disc, macula and vascular arch in fundus photographs. IEEE Trans Med Imaging. 2007;26:116–127.

74. Walter, T, Klein, JC, Massin, P, et al. A contribution of image processing to the diagnosis of diabetic retinopathy – detection of exudates in color fundus images of the human retina. IEEE Trans Med Imaging. 2002;21:1236–1243.

75. Spencer, T, Olson, JA, McHardy, KC, et al. An image-processing strategy for the segmentation and quantification of microaneurysms in fluorescein angiograms of the ocular fundus. Comput Biomed Res. 1996;29:284–302.

76. Frame, AJ, Undrill, PE, Cree, MJ, et al. A comparison of computer based classification methods applied to the detection of microaneurysms in ophthalmic fluorescein angiograms. Comput Biol Med. 1998;28:225–238.

77. Hipwell, JH, Strachan, F, Olson, JA, et al. Automated detection of microaneurysms in digital red-free photographs: a diabetic retinopathy screening tool. Diabet Med. 2000;17:588–594.

78. Niemeijer, M, van Ginneken, B, Staal, J, et al. Automatic detection of red lesions in digital color fundus photographs. IEEE Trans Med Imaging. 2005;24:584–592.

79. Niemeijer, M, van Ginneken, B, Russell, SR, et al. Automated detection and differentiation of drusen, exudates, and cotton-wool spots in digital color fundus photographs for diabetic retinopathy diagnosis. Invest Ophthalmol Vis Sci. 2007;48:2260–2267.

80. Cree, MJ, Olson, JA, McHardy, KC, et al. A fully automated comparative microaneurysm digital detection system. Eye (Lond). 1997;11:622–628.

81. Gardner, GG, Keating, D, Williamson, TH, et al. Automatic detection of diabetic retinopathy using an artificial neural network: a screening tool [see comments]. Br J Ophthalmol. 1996;80:940–944.

82. Sinthanayothin, C, Boyce, JF, Williamson, TH, et al. Automated detection of diabetic retinopathy on digital fundus images. Diabet Med. 2002;19:105–112.

83. Quellec, G, Lamard, M, Cazuguel, G, et al. Adaptive nonseparable wavelet transform via lifting and its application to content-based image retrieval. IEEE Trans Image Process. 2010;19:25–35.

84. Osareh, A, Mirmehdi, M, Thomas, B, et al. Automated identification of diabetic retinal exudates in digital colour images. Br J Ophthalmol. 2003;87:1220–1223.

85. Al-Diri, B, Hunter, A, Steel, D. An active contour model for segmenting and measuring retinal vessels. IEEE Trans Med Imaging. 2009;28:1488–1497.

86. Brinchmann-Hansen, O. The light reflex on retinal arteries and veins. A theoretical study and a new technique for measuring width and intensity profiles across retinal vessels. Acta Ophthalmol Suppl. 1986;179:1–53.

87. Brinchmann-Hansen, O, Sandvik, L. The intensity of the light reflex on retinal arteries and veins. Acta Ophthalmol. 1986;64:547–552.

88. Brinchmann-Hansen, O, Sandvik, L. The width of the light reflex on retinal arteries and veins. Acta Ophthalmol. 1986;64:433–438.

89. Xu, X, Niemeijer, M, Song, Q, et al. Vessel boundary delineation on fundus images using graph-based approach. IEEE Trans Med Imaging. 2011;30:1184–1191.

90. Niemeijer, M, Xu, X, Dumitrescu, A, et al. Automated measurement of the arteriolar-to-venular width ratio in digital color fundus photographs. IEEE Trans Med Imaging. 2011.

91. Lee, S, Reinhardt, JM, Cattin, PC, et al. Objective and expert-independent validation of retinal image registration algorithms by a projective imaging distortion model. Med Image Anal. 2010;14:539–549.

92. Lee, S, Abràmoff, MD, Reinhardt, JM. Retinal image mosaicing using the radial distortion correction model. In: Reinhardt JM, Pluim JP, editors. Medical imaging 2008: image processing. Proc SPIE. 2008;6914:691435–691439.

93. Karnowski, TP, Aykac, D, Chaum, E, et al. Practical considerations for optic nerve location in telemedicine. Conf Proc IEEE Eng Med Biol Soc. 2009;2009:6205–6209.

94. Lamard, M, Cazuguel, G, Quellec, G, et al. Content based image retrieval based on wavelet transform coefficients distribution. Conf Proc IEEE Eng Med Biol Soc. 2007;2007:4532–4535.

95. Niemeijer, M, van Ginneken, B, Cree, MJ, et al. Retinopathy online challenge: automatic detection of microaneurysms in digital color fundus photographs. IEEE Trans Med Imaging. 2010;29:185–195.

96. Hoover, A, Kouznetsova, V, Goldbaum, M. Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans Med Imaging. 2000;19:203–210.

97. Giancardo, L, Abramoff, MD, Chaum, E, et al. Elliptical local vessel density: a fast and robust quality metric for retinal images. Conf Proc IEEE Eng Med Biol Soc. 2008;2008:3534–3537.

98. Messidor. http://latim.univ-brest.fr/index.php?option=com_content&view=article&id=61&Itemid=100034&lang=en, 2005.

99. Tang L, Niemeijer M, Abramoff MD. Splat feature classification: detection of large retinal hemorrhages. 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro. Chicago, IL.

100. Lee, K, Garvin, MK, Russell, S, et al. Automated intraretinal layer segmentation of 3-D macular OCT scans using a multiscale graph search. ARVO Meeting Abstracts. 2010;51:1767.

101. Tang, L, Kwon, YH, Alward, WLM, et al. Automated measurement of optic nerve head shape from stereo color photographs of the optic disc: validation with SD-OCT. ARVO Meeting Abstracts. 2010;51:1774.

102. Antony, BJ, Tang, L, Abramoff, M, et al. Automated method for the flattening of optical coherence tomography images. ARVO Meeting Abstracts. 2010;51:1781.

103. Mahajan, VB, Folk, JC, Russell, SR, et al. Iowa membrane maps: SD OCT guided therapy for epiretinal membrane. ARVO Meeting Abstracts. 2010;51:3604.

104. Verbraak, FD, Van Dijk, HW, Kok, PH, et al. Reduced retinal thickness in patients with type 2 diabetes mellitus. ARVO Meeting Abstracts. 2010;51:4671.

105. Baroni, M, Barletta, G. Contour definition and tracking in cardiac imaging through the integration of knowledge and image evidence. Ann Biomed Eng. 2004;32:688–695.

106. Haeker, M, Abràmoff, MD, Kardon, R, et al. Segmentation of the surfaces of the retinal layer from OCT images. Lecture Notes Computer Sci. 2006;4190:800–807.

107. Haeker, M, Abramoff, MD, Wu, X, et al. Use of varying constraints in optimal 3-D graph search for segmentation of macular optical coherence tomography images. Med Image Comput Comput Assist Interv. 2007;10:244–251.

108. Haeker, M, Wu, X, Abramoff, M, et al. Incorporation of regional information in optimal 3-D graph search with application for intraretinal layer segmentation of optical coherence tomography images. Inf Process Med Imaging. 2007;20:607–618.

109. Garvin, MK, Abramoff, MD, Kardon, R, et al. Intraretinal layer segmentation of macular optical coherence tomography images using optimal 3-D graph search. IEEE Trans Med Imaging. 2008;27:1495–1505.

110. Galler, KE, Folk, JC, Russell, SR, et al. Patient preference and safety of bilateral intravitreal injection of anti-VEGF therapy. ARVO Meeting Abstracts. 2009;50:247.

111. Garvin, MK, Abramoff, MD, Wu, X, et al. Automated 3-D intraretinal layer segmentation of macular spectral-domain optical coherence tomography images. IEEE Trans Med Imaging. 2009;28:1436–1447.

112. Reinhardt, JM, Lee, S, Xu, X, et al. Retina atlas mapping from color fundus images. ARVO Meeting Abstracts. 2009;50:3811.

113. Antony, BJ, Abramoff, MD, Lee, K, et al. Automated 3D segmentation of intraretinal layers from optic nerve head optical coherence tomography images. In: Molthen RC, Weaver JB, editors. Medical Imaging 2010: Biomedical Applications in Molecular, Structural, and Functional Imaging. Proc SPIE. 2010;7626:76260U.

114. Van Dijk, HW, Kok, PH, Garvin, M, et al. Selective loss of inner retinal layer thickness in type 1 diabetic patients with minimal diabetic retinopathy. Invest Ophthalmol Vis Sci. 2009;50:3404–3409.

115. van Dijk, HW, Verbraak, FD, Kok, PH, et al. Decreased retinal ganglion cell layer thickness in patients with type 1 diabetes. Invest Ophthalmol Vis Sci. 2010;51:3660–3665.

116. Longmuir, SQ, Longmuir, R, Matthews, K, et al. Retinal arterial but not venous tortuosity correlates with facioscapulohumeral muscular dystrophy (FSHD) severity. ARVO Meeting Abstracts. 2009;50:5419.

117. Quellec, G, Lamard, M, Cazuguel, G, et al. Multimodal information retrieval to assist diabetic retinopathy diagnosis. ARVO Meeting Abstracts. 2009;50:1363.

118. Agurto Rios, C, Pattichis, MS, Murillo, S, et al. Detection of structures in the retina using AM-FM for diabetic retinopathy classification. ARVO Meeting Abstracts. 2009;50:313.

119. Tso, DY, Schallek, JB, Kardon, R, et al. Hemodynamic components contribute to intrinsic signals of the retina and optic disc. ARVO Meeting Abstracts. 2009;50:4322.

120. Garvin, MK, Niemeijer, M, Kardon, RH, et al. automatically correcting for the presence of retinal vessels on spectral-domain optical coherence tomography images decreases variability of the segmented retinal layers. ARVO Meeting Abstracts. 2009;50:1099.

121. Lee, K, Niemeijer, M, Garvin, MK, et al. Automated optic disc segmentation from 3D SD-OCT of the optic nerve head (ONH). ARVO Meeting Abstracts. 2009;50:1102.

122. Xu, Y, Sonka, M, McLennan, G, et al. MDCT-based 3-D texture classification of emphysema and early smoking related lung pathologies. IEEE Trans Med Imaging. 2006;25:464–475.

123. Quellec, G, Lee, K, Dolejsi, M, et al. Three-dimensional analysis of retinal layer texture: identification of fluid-filled regions in SD-OCT of the macula. IEEE Trans Med Imaging. 2010;29:1321–1330.

124. Van Dijk, HW, Kok, PHB, Garvin, M, et al. Selective loss of inner retinal layer thickness in type 1 diabetic patients with minimal diabetic retinopathy. ARVO Meeting Abstracts. 2009;50:3244.

125. Abramoff, MD, Russell, SR, Mahajan, V, et al. Performance of automated detection of diabetic retinopathy does not improve by using the distance of each lesion to the fovea. ARVO Meeting Abstracts. 2009;50:3268.

126. Barriga, ES, Russell, SR, Pattichis, MS, et al. Relationship between visual features and analytically derived features in non-exudated AMD phenotypes: closing the sematic gap. ARVO Meeting Abstracts. 2009;50:3274.

127. Hu, Z, Niemeijer, M, Lee, K, et al. Automated segmentation of the optic canal in 3D spectral-domain OCT of the optic nerve head (ONH) using retinal vessel suppression. ARVO Meeting Abstracts. 2009;50:3334.

128. Niemeijer, M, Garvin, MK, Lee, K, et al. Automated segmentation of the retinal vasculature silhouettes in isotropic 3D optical coherence tomography scans. ARVO Meeting Abstracts. 2009;50:1103.

129. Niemeijer, M, Garvin, MK, van Ginneken, B, et al. Vessel segmentation in 3D spectral OCT scans of the retina. Medical Imaging 2008: Image Processing. 2008;6914:69141R–8.

130. Niemeijer, M, Sonka, M, Garvin, MK, et al. Automated segmentation of the retinal vasculature in 3D optical coherence tomography images. ARVO Meeting Abstracts. 2008;49:1832.

131. Davis, B, Russell, SR, Abramoff, MD, et al. Application of independent component analysis to classify non-exudative amd phenotypes quantitatively from vision-based derived features in retinal images. ARVO Meeting Abstracts. 2008;49:887.

132. Xu, J, Ishikawa, H, Wollstein, G, et al. Retinal vessel segmentation on SLO image. Conf Proc IEEE Eng Med Biol Soc. 2008;2008:2258–2261.

133. Nelson, D, Abramoff, MD, Kwon, YH, et al. Correlation of progression between structure and function in ocular hypertensive patients from the SAFE study. ARVO Meeting Abstracts. 2008;49:738.

134. Lee, K, Abramoff, MD, Niemeijer, M, et al. 3-D segmentation of retinal blood vessels in spectral-domain OCT volumes of the optic nerve head. In: Molthen RC, Weaver JB, editors. Medical Imaging 2010: Biomedical Applications in Molecular, Structural, and Functional Imaging. Proc SPIE. 2010;7626:76260V.

135. Duda, RA, Hart, PE, Stork, DG. Pattern classification. New York: Wiley-Interscience; 2001.

136. Lee, S, Reinhardt, JM, Niemeijer, M, et al. Comparing the performance of retinal image registration algorithms using the centerline error measurement metric. ARVO Meeting Abstracts. 2008;49:1833.

137. Garvin, MK, Sonka, M, Kardon, RH, et al. Three-dimensional analysis of SD OCT: thickness assessment of six macular layers in normal subjects. ARVO Meeting Abstracts. 2008;49:1879.

138. Boykov, Y, Kolmogorov, V. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans Pattern Anal Mach Intell. 2004;26:1124–1137.

139. Yang, EB, Jin Jones, Y, Alward, WLM, et al. Comparing resident and fellow performance on evaluation of stereoscopic optic disc images. ARVO Meeting Abstracts. 2008;49:3626.

140. Tobin, KW, Chaum, E, Abramoff, MD, et al. Automated diagnosis of retinal disease in a large diabetic population. ARVO Meeting Abstracts. 2008;49:3225.

141. Graff, JM, Abramoff, MD, Russell, SR. Stereo video indirect ophthalmoscopy: a novel educational and research tool. ARVO Meeting Abstracts. 2008;49:3228.

142. Abramoff, MD, Kardon, RH, Vermeer, KA, et al. A portable, patient friendly scanning laser ophthalmoscope for diabetic retinopathy imaging: exudates and hemorrhages. ARVO Meeting Abstracts. 2007;48:2592.

143. Russell, SR, Abramoff, MD, Radosevich, MD, et al. Quantitative assessment of retinal image quality compared to subjective determination. ARVO Meeting Abstracts. 2007;48:2607.

144. Piette, SD, Adix, ML, Abramoff, MD, et al. Comparison of computer aided planimetry between simultaneous and non-simultaneous stereo optic disc photographs. ARVO Meeting Abstracts. 2007;48:1184.

145. Lee, S, Abramoff, MD, Reinhardt, J. Feature-based pairwise retinal image registration by radial distortion correction. Med Imaging: Image Anal Proc SPIE Med Imaging. 2007:6512.

146. Tso, DY, Schallek, J, Kwon, Y, et al. Blood flow dynamics contribute to functional intrinsic optical signals in the cat retina in vivo. ARVO Meeting Abstracts. 2007;48:1951.

147. Lee, S, Abramoff, MD, Reinhardt, JM. Retinal atlas statistics from color fundus images. In: Dawant BM, Haynor DR, editors. Medical Imaging 2010: Image Processing. Proc SPIE. 2010;7623:762310–762319.

148. Vermeer, KA, Mensink, MH, Kardon, RH, et al. Super resolution in retinal imaging. ARVO Meeting Abstracts. 2007;48:2766.

149. Niemeijer, M, Garvin, MK, Lee, K, et al. Registration of 3D spectral OCT volumes using 3D SIFT feature point matching. Med Imaging 2009: Image Processing. 2009;7259:72591I–8.

150. Lowe, DG. Distinctive image features from scale-invariant keypoints. Int J Comput Vision. 2004;60:91–110.

151. Klein, R, Chou, CF, Klein, BE, et al. Prevalence of age-related macular degeneration in the US population. Arch Ophthalmol. 2011;129:75–80.

152. Sánchez, CI, Hornero, R, Mayo, A, et al. Mixture model-based clustering and logistic regression for automatic detection of microaneurysms in retinal images. In: Karssemeijer N, Giger ML, editors. Medical Imaging 2009: Computer-Aided Diagnosis. Proc SPIE. 2009;7260:72601M.

153. Cree, MJ, The Waikato Microaneurysm Detector Univ. Waikato, Tech. Rep., 2008. http://roc.healthcare.uiowa.edu/results/documentation/waikato.pdf.

154. Quellec, G, Lamard, M, Josselin, PM, et al. Optimal wavelet transform for the detection of microaneurysms in retina photographs. IEEE Trans Med Imag. 2008;27:1230–1241.

155. Zhang, B, Wu, X, You, J, et al. Hierarchical detection of red lesions in retinal images by multiscale correlation filtering. In: Karssemeijer N, Giger ML, editors. Medical Imaging 2009: Computer-Aided Diagnosis. Proc SPIE. 2009;7260:72601L.

156. Mizutani, A, Muramatsu, C, Hatanaka, Y, et al. Automated microaneurysm detection method based on double ring filter in retinal fundus images. In: Karssemeijer N, Giger ML, editors. Medical Imaging 2009: Computer-Aided Diagnosis. Proc SPIE. 2009;7260:72601N.