Case Study

After studying the material in this chapter, the reader should be able to respond to the following case study:

On an 8:00 am assay run, the results for three levels of a preserved hemoglobin control sample are 2 g/dL higher than the upper limit of the target interval. The medical laboratory scientist reviews δ-check data on the last 10 patient results and notices that the results are consistently 1.8 to 2.2 g/dL higher than results generated the previous day.

In clinical laboratory science, quality implies the ability to provide accurate, reproducible assay results that offer clinically useful information.¹ Because physicians base 70% of their clinical decision making on laboratory results, the results must be reliable. Reliability requires vigilance and effort on the part of all laboratory staff members, and this effort is often directed by an experienced medical laboratory scientist who is a quality assurance and quality control specialist.

Of the terms quality control and quality assurance, quality assurance is the broader concept, encompassing preanalytical, analytical, and postanalytical variables (Box 5-1).² Quality control processes document assay validity, accuracy, and precision, and should include external quality assessment, reference interval preparation and publication, and lot-to-lot validation.³

Preanalytical variables are addressed in Chapter 3, which discusses blood specimen collection, and in Chapter 45, which includes a section on coagulation specimen management. Postanalytical variables are discussed at the end of this chapter.

The control of analytical variables begins with laboratory assay validation.

Validation of a New or Modified Assay

All new laboratory assays and all assay modifications require validation.⁴ Validation is comprised of procedures to determine accuracy, specificity, precision, limits, and linearity.⁵ The results of these procedures are faithfully recorded and made available to on-site assessors upon request.⁶

Accuracy

Accuracy is the measure of concurrence or difference between an assay value and the theoretical “true value” of an analyte (Figure 5-1). Some statisticians prefer to define accuracy as the extent of error between the assay result and the true value. Accuracy is easy to define but difficult to establish and maintain.

For many analytes, laboratory scientists employ primary standards to standardize assays and establish accuracy. A primary standard is a material of known, fixed composition that is prepared in pure form, often by weighing on an analytical balance. The scientist dissolves the weighed standard in an aqueous solution, prepares suitable dilutions, calculates the anticipated concentration in each dilution, and assigns the calculated concentrations to assay outcomes. For example, the scientist may obtain pure glucose, weigh 100 mg, dilute it in 100 mL of buffer, and assay an aliquot of the solution using photometry. The resulting absorbance would then be assigned the value of 100 mg/dL. The scientist may repeat this procedure using a series of four additional glucose solutions at 20, 60, 120, and 160 mg/dL to produce a five-point “standard curve.” The curve may be re-assayed several times to generate means for each concentration. The assay is then employed on human plasma, with absorbance compared with the standard curve to generate a result. The matrix of a primary standard need not match the matrix of the patient specimen; the standard may be dissolved in an aqueous buffer, whereas the test specimen may be human serum or plasma.

To save time and resources, the scientist may employ a secondary standard, perhaps purchased, that the vendor has previously calibrated to a primary standard. The secondary standard may be a preserved plasma preparation delivered at a certified known concentration. The scientist merely thaws or reconstitutes the secondary standard and incorporates it into the test series during validation or revalidation. Manufacturers often match secondary standards as closely as possible to the test sample’s matrix, for instance, plasma to plasma, whole blood to whole blood. Neither primary nor secondary standards are assayed during routine patient sample testing, only during calibration.

Unfortunately, in hematology and hemostasis, in which the analytes are often cell suspensions or enzymes, there are just a handful of primary standards: cyanmethemoglobin, fibrinogen, factor VIII, protein C, antithrombin, and von Willebrand factor.⁷ For scores of analytes, the hematology and hemostasis laboratory scientist relies on calibrators. Calibrators for hematology may be preserved human blood cell suspensions, sometimes supplemented with microlatex particles or nucleated avian red blood cells (RBCs) as surrogates for hard-to-preserve human white blood cells (WBCs). In hemostasis, calibrators may be frozen or lyophilized plasma from healthy human donors. For most analytes it is impossible to prepare “weighed-in” standards; instead, calibrators are assayed using reference methods (“gold standards”) at selected independent expert laboratories. For instance, a vendor may prepare a 1000-L lot of preserved human blood cell suspension, assay for the desired analytes in house, and send aliquots to five laboratories that employ well-controlled reference instrumentation and methods. The vendor obtains blood count results from all five, averages the results, compares them to the in-house values, and publishes the averages as the reference calibrator values. The vendor then distributes sealed aliquots to customer laboratories with the calibrator values published in the accompanying package inserts. Vendors often market calibrators in sets of three or five, spanning the range of assay linearity or the range of potential clinical results.

As with secondary standards, vendors attempt to match their calibrators as closely as possible to the physical properties of the test sample. For instance, human preserved blood used to calibrate complete blood count analytes is prepared to closely match the matrix of fresh anticoagulated patient blood specimens, despite the need for preservatives, refrigeration, and sealed packaging. Vendors submit themselves to rigorous certification by governmental or voluntary standards agencies in an effort to verify and maintain the validity of their products.

The scientist assays the calibration material using the new or modified assay and compares results with the vendor’s published results. When new results parallel published results within a selected range, for example ±10%, the results are recorded and the assay is validated for accuracy. If they fail to match, the new assay is modified or a new reference interval and therapeutic range is prepared.

Medical laboratory scientists may employ locally collected fresh normal blood as a calibrator; however, the process for validation and certification is laborious, so few attempt it. The selected specimens are assayed using reference equipment and methods, calibration values are assigned, and the new or modified assay is calibrated from these values. The Student t-test is often the statistic employed to match the means of the reference and of the new assay. Often the reference equipment and methods are provided by a nearby laboratory.

Determination of Accuracy by Regression Analysis

If a series of five calibrators is used, results may be analyzed by the following regression equation:

< ?xml:namespace prefix = "mml" />y=a+bx

Slope(b)=[n∑XY−(∑X)(∑Y)]/[n∑X2−(∑X)2]

Intercept(a)=[∑Y−b(∑X)]/n

where x and y are the variables; a = intercept between the regression line and the y-axis; b = slope of the regression line; n = number of values or elements; X = first score; Y = second score; ΣXY = sum of the product of first and second scores; ΣX = sum of first scores; ΣY = sum of second scores; ΣX² = sum of squared first scores. Perfect correlation generates a slope of 1 and a y intercept of 0. Local policy establishes limits for slope and y intercept; for example, many laboratory directors reject a slope of less than 0.9 or an intercept of more than 10% above or below zero (Figure 5-2).

Slope measures proportional systematic error; the higher the analyte value, the greater the deviation from the line of identity. Proportional errors are caused by malfunctioning instrument components or a failure of some part of the testing process. The magnitude of the error increases with the concentration or activity of the analyte. An assay with proportional error may be invalid.

Intercept measures constant systematic error (or bias, in laboratory vernacular), a constant difference between the new and reference assay regardless of assay result magnitude. A laboratory director may choose to adopt a new assay with systematic error, but must modify the published reference interval.

Regression analysis gains sufficient power when 100 or more patient specimens are tested using both the new and reference assay in place of or in addition to calibrators. Data may be entered into a spreadsheet program that offers an automatic regression equation.

Precision

Unlike determination of accuracy, assessment of precision (dispersion, reproducibility, variation, random error) is a simple validation effort, because it merely requires performing a series of assays on a single sample or lot of reference material.⁸ Precision studies always assess both within-day and day-to-day variation about the mean and are usually performed on three to five calibration samples, although they may also be performed using a series of patient samples. To calculate within-day precision, the scientist assays a sample at least 20 consecutive times using one reagent batch and one instrument run. For day-to-day precision, at least 10 runs on 10 consecutive days are required. The day-to-day precision study employs the same sample source and instrument but separate aliquots. Day-to-day precision accounts for the effects of different operators, reagents, and environmental conditions such as temperature and barometric pressure.

The collected data from within-day and day-to-day sequences are reduced by formula to the mean and a measure of dispersion such as standard deviation or, most often, coefficient of variation in percent (CV%):

Mean(x¯)=∑χn;

where (Σx) = the sum of the data values and n = the number of data points collected

Standarddeviation(s)=∑(x−x¯)2n−1

CV%=100sx¯

The CV% documents the degree of random error generated by an assay, a function of assay stability.

CV% limits are established locally. For analytes based on primary standards, the within-run CV% limit may be 5% or less, and for hematology and hemostasis assays, 10% or less; however, the day-to-day run CV% limits may be as high as 30%, depending upon the complexity of the assay. Although accuracy, linearity, and analytical specificity are just as important, medical laboratory scientists often equate the quality of an assay with its CV%. The best assay, of course, is one that combines the lowest CV% with the greatest accuracy.

Precision for visual light microscopy leukocyte differential counts on stained blood films is immeasurably broad, particularly for low-frequency eosinophils and basophils.⁹ Most visual differential counts are performed by reviewing 100 to 200 leukocytes. Although impractical, it would take differential counts of 800 or more leukocytes to improve precision to measurable levels. Automated differential counts generated by profiling instruments, however, provide CV% levels of 5% or lower because these instruments count thousands of cells.

Linearity

Linearity is the ability to generate results proportional to the calculated concentration or activity of the analyte. The laboratory scientist dilutes a high-end calibrator or patient sample to produce at least five dilutions spanning the full range of assay. The dilutions are then assayed. Computed and assayed results for each dilution are paired and plotted on a linear graph, x scale and y scale, respectively. The line is inspected visually for nonlinearity at the highest and lowest dilutions (Figure 5-3). The acceptable range of linearity is established inboard based on the values at which linearity loss is evident. Although formulas exist for computing the limits of linearity, visual inspection is the accepted practice. Nonlinear graphs may be transformed using semilog or log-log graphs when necessary.

Patient samples with results above the linear range must be diluted and reassayed. Results from diluted samples that fall within the linear range are valid; however, they must be multiplied by the dilution. Laboratory scientists never report results that fall below or above the linear limits, because accuracy is compromised in the nonlinear regions of the assay. Lower limits are especially important when counting platelets or assaying coagulation factors. For example, the difference between 1% and 3% factor VIII activity affects treatment options and the potential for predicting coagulation factor inhibitor formation. Likewise, the difference between a platelet count of 10,000/mcL and 5000/mcL affects the decision to treat with platelet concentrate.

Limits

Linearity studies are coupled with the lower limit of detection study. A “zero calibrator,” or blank, is assayed 20 times and the standard deviation is computed from the results. The lower limit of detection is determined from the computed standard deviation. The limit is three standard deviations above the mean of blank assays. This cutoff prevents false-positive results generated by low-end assay interference, commonly called noise. Limit assays are typically performed by the manufacturer or distributor and the results provided on the package insert; however, local policies often require that results of the manufacturer’s limit studies be confirmed.

Analytical Specificity

Analytical specificity is the ability of an assay to distinguish the analyte of interest from anticipated interfering substances within the sample matrix. The laboratory scientist “spikes” identical samples with potential interfering substances and measures the effects of each upon the assay results. Specificity is always determined by the manufacturer and need not be confirmed at the local provider’s site unless there is suspicion of interference from a particular substance not assayed by the manufacturer. Manufacturer specificity data are transferred from the package insert to the local validation report.

Levels of Laboratory Assay Approval

The U.S. Food and Drug Administration (FDA) categorizes assays as cleared, analyte-specific reagent (ASR) assays, research use only (RUO) assays, and home-brew assays. FDA-cleared assays are approved for the detection of specific analytes and should not be used for off-label applications. Details are given in Table 5-1.

TABLE 5-1

Categories of Laboratory Assay Approval

Assay Category	Comment
Food and Drug Administration cleared	The local institution may use package insert data for linearity and specificity but must establish accuracy and precision.
Analyte-specific reagent	Manufacturer may provide individual reagents but not in kit form, and may not provide package insert validation data. Local institution must perform all validation steps.
Research use only	Local institution must perform all validation steps. Research use only assays are intended for clinical trials, and carriers are not required to pay.
Home brew	Assays devised locally, all validation studies are performed locally.

Documentation and Reliability

Validation is recorded on standard forms. The Clinical Laboratory Standards Institute (CLSI) and David G. Rhoads Associates (http://www.dgrhoads.com/files1/EE5SampleReports.pdf) provide automated electronic forms. Validation records are stored in prominent laboratory locations and made available to laboratory assessors upon request.

Precision and accuracy maintained over time provide assay reliability. The recalibration interval may be once every 6 months or in accordance with operators’ manual recommendations. Recalibration is necessary whenever reagent lots are updated unless the laboratory can demonstrate that the reportable range is unchanged using lot-to-lot comparison. When control results demonstrate a shift or consistently fall outside action limits, or when an instrument is repaired, the validation procedure is repeated.

Regularly scheduled validity rechecks, lot-to-lot comparisons, instrument preventive maintenance, staff competence, and scheduled performance of internal quality control and external quality assessment procedures assure continued reliability and enhance the value of a laboratory assay to the patient and physician.

Lot-to-Lot Comparisons

Laboratory managers reach agreements with vendors to sequester kit and reagent lots, which ensures infrequent lot changes, optimistically no more than once a year. The new reagent lot must arrive approximately a month before the laboratory runs out of the old lot so that lot-to-lot comparisons may be completed and differences resolved, if necessary. The scientist uses control or patient samples and prepares a range of analyte dilutions, typically five, spanning the limits of linearity. If the reagent kits provide controls, these are also included, and all are assayed using the old and new reagent lots. Results are charted as illustrated in Table 5-2.

Action limits vary by location, but many managers reject the new lot when more than one sample generates a variance greater than 10% or when all variances are positive or negative. In the latter case, the new lot may be rejected or it may be necessary to use the lot but develop a new reference interval and therapeutic range.

Development of the Reference Interval and Therapeutic Range

Once an assay is validated, the laboratory scientist develops the reference interval (reference range, normal range).¹⁰ Most practitioners tend to use the vernacular term normal range; however, reference interval is the preferred term. According to the mathematical definitions, “range” encompasses all assay results from largest to smallest, whereas “interval” is a statistic that trims outliers.

To develop a reference interval the scientist carefully defines the desired normal population and recruits representative donors who meet the criteria to provide blood specimens. The definition may, for example, exclude smokers, women taking oral contraceptives, and people using over-the-counter or prescription medications. Donors may be paid. There should be an equal number of males and females, and the normal subjects should match the institution’s population demographics in terms of age and race. When practical, large-volume blood specimens are collected, aliquoted, and placed in long-term storage. For instance, plasma aliquots for coagulation reference interval development are stored at –70° C. It may be impractical to develop local reference intervals for infants, children, or geriatric populations. In these cases the laboratory director may choose to use published (textbook) intervals. In general, although published normal values are available for educational and general discussion purposes, local laboratories must generate their own reference intervals to most closely match the demographics of the area served by their institution.

The minimum number of subject samples required to develop a reference interval may be determined using statistical power computations; however, practical limitations prevail.¹¹ For a completely new assay with no currently established reference interval, a minimum of 120 samples is necessary. In most cases, however, the assay manufacturer provides a reference interval on the package insert, and the local laboratory scientist need only assay 30 samples, 15 male and 15 female, to validate the manufacturer’s reference interval, a process called transference. Likewise, the scientist may refer to published reference intervals and, once they are locally validated, transfer them to the institution’s report form.

It is assumed that the population sample subjects employed to generate reference intervals will produce frequency distributions (in laboratory vernacular, histograms) that are normal bell-shaped (gaussian) curves (Figure 5-4). In a gaussian frequency distribution the mean is at the center and the dispersion about the mean is identical in both directions. In many instances, however, biologic frequency distributions are “log-normal” with a “tail” on the high end. For example, it has been assumed for years that the visual reticulocyte percentage reference interval in adults is 0.5% to 1.5%; however, repeated analysis of normal populations in several locations has established it at 0.5% to 2%, owing to a subset of normal subjects whose reticulocyte counts fall at the high end of the range. Scientists may choose to live with a log-normal distribution or they may transform it by replotting the curve using a semilog or log-log graphic display. The decision to transform may arise locally but eventually becomes adopted as a national practice standard.

In a normal distribution, the mean () is computed by dividing the sum of the observed values by the number of data points, n, as shown in the equation on page 45. The standard deviation is calculated using the second formula in that equation. A typical reference interval is computed as ±2 standard deviations and assumes that the distribution is normal. The limits at ±2 standard deviations encompass 95.46% of normal results, known as the 95% confidence interval. This implies that 4.54% of theoretically normal results fall outside the interval. A standard deviation computed from a nongaussian distribution may turn out to be too narrow to reflect the true reference interval and may thus encompass fewer than 95% of theoretical normal values and generate a number of false positives. Assays with high CV% values have high levels of random error reflected in a broad curve; low CV% assays with “tight” dispersal have smaller random error and generate a narrow curve, as illustrated in Figure 5-1. The breadth of the curve may also reflect biologic variation in values of the analyte.

A few hematology and hemostasis assays are used to monitor drug therapy. For instance, the international normalized ratio (INR) for prothrombin time is used to monitor the effects of oral Coumadin (warfarin) therapy, and the therapeutic range is universally established at an INR of 2 to 3. On the other hand, the therapeutic range for monitoring treatment with unfractionated heparin using the partial thromboplastin time (PTT) assay must be established locally by performing regression of the PTT results in seconds against the results of the chromogenic anti-Xa heparin assay, whose therapeutic range is established empirically as 0.3 to 0.7 international heparin units. The PTT therapeutic range is called the Brill-Edwards curve and is described in Chapter 45.

If assay revalidation or lot-to-lot comparison reveals a systematic change caused by reagent or kit modifications, a new reference interval (and therapeutic range, when applicable) is established. The laboratory director must warn the hospital staff of reference interval and therapeutic range changes, because failure to observe new intervals and ranges may result in diagnosis and treatment errors.

Internal Quality Control

Controls

Laboratory managers prepare, or more often purchase, assay controls. Although it may appear similar, a control is wholly distinct from a calibrator. Indeed, cautious laboratory directors may insist that controls be purchased from distributors different from those who supply their calibrators. As discussed in the section Validation of a New or Modified Assay, calibrators are used to adjust instrumentation or to develop a standard curve. Calibrators are assayed by a reference method in expert laboratories and their assigned value is certified. Controls are used independently of the calibration process so that systematic errors caused by deterioration of the calibrator or a change in the analytical process can be detected through internal quality control. This process is continuous and is called calibration verification.¹² Compared with calibrators, control materials are inexpensive and are comprised of the same matrix as patient samples except for preservatives or freezing that provide a long shelf life. Controls provide known values and are sampled directly alongside patient specimens to accomplish within-run assay validation. In nearly all instances, two controls are required per test run, one in the normal range and the other in an expected abnormal range. For some assays there is reason to select controls whose values are near the interface of normal and abnormal. In institutions that perform continuous runs, the controls should be run at least once per shift, for instance, at 7 am, 3 pm, and 11 pm. In laboratories where assay runs are discrete events, two controls are assayed with each run.

Control results must fall within predetermined dispersal limits, typically ±2 standard deviations. Control manufacturers provide limits; however, local laboratory scientists must validate and transfer manufacturer limits or establish their own, usually by computing standard deviation from the first 20 control assays. Whenever the result for a control is outside the limits the run is rejected and the cause is found and corrected. The steps for correction are listed in Table 5-3.

Control results are plotted on a Levey-Jennings chart that displays each data point in comparison to the mean and limits (Figure 5-5).¹³ The Levey-Jennings chart assumes that the control results distribute in a gaussian manner and provide limits at 1, 2, and 3 standard deviations about the mean. In addition to being analyzed for single-run errors, the data points are examined for sequential errors over time (Figure 5-6). Both single-run and long-term control variation are a function of assay dispersion or random error and reflect the CV% of an assay.

Dr. James Westgard has established a series of internal quality control rules that are routinely applied to long-term deviations, called the Westgard rules.¹⁴ The rules were developed for assays that employ primary standards, but a few Westgard rules that are the most useful in hematology and hemostasis laboratories are provided in Table 5-4, along with the appropriate actions to be taken.¹⁵

Moving Average () of the Red Blood Cell Indices

In 1974, Dr. Brian Bull proposed a method of employing patient RBC indices to monitor the stability of hematology analyzers, recognizing that the RBC indices mean cell volume (MCV), mean cell hemoglobin (MCH), and mean cell hemoglobin concentration (MCHC) remain constant on average despite individual patient variations.¹⁶ Each consecutive sequence of 20 patient RBC index assay results is collected and treated by the moving average formula, which accumulates, “smoothes,” and trims data to reduce the effect of outliers. Each trimmed 20-sample mean, , is plotted on a Levey-Jennings chart and tracked for trends and shifts using Westgard rules. The formula has been automated and embedded in the circuitry of all hematology analyzers, which provide a Levey-Jennings graph for MCV, MCH, and MCHC. The moving average concept has been generalized to WBC and platelet counts and to some clinical chemistry analytes, albeit with moderate success.

To begin, 500 consecutive samples are analyzed for the mean MCV, MCH, and MCHC. A Levey-Jennings chart is prepared using ±3% of the mean or one standard deviation as the action limits, and subsequent data accumulation commences in groups of 20.

The moving average method requires a computer to calculate the averages, does not detect within-run errors, and is less sensitive than the use of commercial controls in detecting systematic shifts and trends. It works less well in institutions that assay samples from specialized populations, such as sickle cell or oncology populations, for which the index values may include a number of outliers. It does not replace the use of control samples but provides additional protection against shifts and trends.

Delta Checks

The δ-check system compares a current analyte result with the result from the most recent previous analysis for the same patient.¹⁷ Patient values remain relatively consistent over time unless there is an intervention. A result that fails a δ check is investigated for intervention or a profound change in the patient’s condition subsequent to the previous analysis. If there is no ready explanation, the failed δ check may indicate an analytical error or mislabeled specimen. Results that fail a δ check are sequestered until the cause is found. Not all assays are amenable to δ checking; only analytes known to show little intraindividual variation are checked. These include MCV and RBC distribution width. In hemostasis, the prothrombin time and INR are checked. Action limits for δ checks are based on clinical impression and are assigned by hematology and hemostasis laboratory directors in collaboration with the house and laboratory staff. Computerization is essential, and δ checks are designed only to identify gross errors, not changes in random error, or shifts or trends. There is no regulatory requirement for δ checks.

External Quality Assessment

External quality assessment further validates the accuracy of hematology and hemostasis assays by comparing results from identical aliquots of samples distributed at regular intervals among laboratories nationwide or worldwide. The aliquots are often called survey or proficiency testing samples and include preserved human donor plasma and whole blood, stained blood and bone marrow smears, and photomicrographs of cells or tissues.

In most proficiency testing systems, target (true or reference) values for the test samples are established in-house by their manufacturer or distributor and are then further validated by preliminary distribution to a handful of “expert” laboratories. Separate target values may be assigned for various assay methods and instruments, as feasible.

Laboratories that participate in external quality assessment are directed to manage the survey samples using the same sample-handling principles as those employed for patient specimens—survey samples should not receive special attention. Turnaround is swift, and results are sent electronically to the provider.

In addition to establishing a target value, agencies that administer surveys reduce the returned data to statistics, including the mean, median, and standard deviation of all participant results. Provided the survey is large enough, the statistics may be computed individually for the various instruments and assay methods. The statistics should match the predetermined targets. If they do not, the agency troubleshoots the assay and assigns the most reliable statistics, usually the group mean and standard deviations.

The agency provides a report to each laboratory, illustrating its result in comparison with the target value and appending a comment if the laboratory result exceeds the established limits, usually ±2 standard deviations from the mean. If the specimen is a blood or bone marrow smear, a photomicrograph, or a problem that requires a binary (positive/negative, yes/no) response, the local laboratory comment is compared with expert opinion and consensus.

Although a certain level of error is tolerated, error rates that exceed established limits result in corrective action or, in extreme circumstances, loss of laboratory accreditation or licensure.

There are a number of external quality assessment agencies; however, the College of American Pathologists (CAP) provides the largest survey systems, CAP accreditation, and CAP Surveys/EXCEL proficiency testing. Survey packages are provided for laboratories offering all levels of service. CAP is a nongovernmental agency; however, survey participation is necessary to meet the accreditation requirements of the Joint Commission (formerly the Joint Commission on Accreditation of Healthcare Organizations) and to qualify for Medicare reimbursement. The North American Specialized Coagulation Laboratory Association provides survey systems for specialty coagulation laboratories in the United States and Canada, and is affiliated with the ECAT (external quality control of diagnostic assays and tests) Foundation External Quality Assessment Program of The Netherlands, which provides survey materials throughout Europe. Many state health agencies provide proficiency testing surveys, requiring laboratories to participate as a condition of licensure.

Measurement of Clinical Efficacy

Since 1940 and before, surgeons have used the bleeding time test to predict the risk of intraoperative hemorrhage. The laboratory scientist or phlebotomist activates an automated lancet to make a 5-mm long, 1-mm deep incision in the volar surface of the forearm and, using a filter paper to soak up the blood, times the interval from the initial incision to bleeding cessation, normally 2 to 9 minutes. The test is simple and logical, and for over 50 years experts have claimed that if the incision bleeds for longer than 9 minutes, there is a risk of surgical bleeding. In the 1990s clinical researchers compared normal and prolonged bleeding times with instances of intraoperative bleeding and found to their surprise that prolonged bleeding time predicted only 50% of intraoperative bleeds.18,19 The other 50% occurred despite a normal bleeding time. Thus the positive predictive value of the bleeding time for intraoperative bleeding was 50%, the same as the probability of obtaining heads (or tails) in tossing a coin. Today the bleeding time test is widely agreed to have no clinical relevance and is obsolete.

Like the bleeding time test, many time-honored hematology and hemostasis assays gain credibility on the basis of expert opinion. Now, however, besides being valid, accurate, linear, and precise, a new or modified assay must be clinically effective.²⁰ To compute clinical efficacy, the scientist uses a series of samples from normal healthy subjects, called controls, and patients who conclusively possess a disease or condition. The patient’s diagnosis is based on clinical outcomes, discharge diagnosis notes, or the results of existing laboratory tests, excluding the new assay. The new assay is then applied to samples from both the normal and disease groups to assess its efficacy.

In a perfect world, the laboratory scientist sets the discrimination threshold at the 95% confidence interval limit (±2 standard deviations) of the reference interval. When this threshold is used, the test will hopefully yield a positive result (e.g., elevated or reduced level) in every instance of disease and a negative result (within the reference interval) in all normal control subjects. In reality, there is always overlap: a “gray area” in which some positive test results are generated by normal samples (false positives) and some negative results are generated by samples from patients with disease (false negatives). False positives cause unnecessary anxiety, follow-up expense, and erroneous diagnostic leads—worrisome, expensive, and time consuming, but not fatal. False negatives fail to detect the disease and may delay treatment, which is potentially life-threatening. The laboratory scientist employs clinical efficacy computations to establish laboratory assay efficacy and minimize both false positives and false negatives (Table 5-5). Clinical efficacy testing includes determination of diagnostic sensitivity and specificity, and positive and negative predictive value, as well as receiver operating characteristic analysis.

To start a clinical efficacy study the scientist selects normal control samples and samples from people proven to have the disease or condition addressed by the assay. To make the discussion easy, assume that 50 samples of each are chosen. All are assayed, and the results turn out as shown in Table 5-6.

The scientist next computes clinical sensitivity and specificity and positive and negative predictive value as shown in Table 5-7.

For all assays, as sensitivity rises, specificity decreases. Assays with high sensitivity and low specificity make effective screening tests, although they produce a number of false positives. For instance, if the condition being studied has a prevalence of 0.0001 (1 in 10,000) and the false-positive rate is 1%, the assay will produce 99 false-positive results for every true-positive result. Clearly such a test is useful only when the consequence of a false-positive result is minimal.

Likewise, as specificity rises, sensitivity goes down. Assays with high specificity make effective confirmatory assays when used in follow-up to positive results on screening assays. High-specificity assays produce a number of false negatives and should not be used as initial screens. A positive result on both a screening assay and a confirmatory assay provides a definitive conclusion. A positive screening result followed by a negative confirmatory test result generates a search for alternative diagnoses.

Laboratory assays are most effective when used for patients with high clinical pretest probability. In such instances, the prevalence of the condition is high enough to mitigate the effects of false positives and false negatives. When a physician orders hemostasis testing for patients who are experiencing easy bruising, there is a high pretest probability, which raises the tests’ clinical efficacy. Ordering hemostasis tests routinely for healthy individuals prior to elective surgery presumes a low pretest probability and reduces the efficacy of the test profile.

Receiver Operating Characteristic Curve

A receiver operating characteristic (ROC) curve or ROC analysis is a further refinement of clinical efficacy testing that may be employed when the assay being validated generates a continuous variable.²¹ In clinical efficacy testing as described earlier, the ±2 standard deviation limit of the reference interval is used as the threshold for discriminating a positive from a negative test result. Often the “true” threshold varies from the ±2 standard deviation limit. Using ROC analysis, the discrimination threshold is adjusted by increments of 1, and the true-positive and false-positive rates are recomputed for each new threshold level. The threshold that is selected is the one that provides the largest true-positive and smallest false-positive rate (Figure 5-7). A line graph is generated plotting true positives on the y-axis and false positives on the x-axis. The overall efficacy of the assay is assessed by measuring the area under the curve. If the area under the curve is 0.5, the curve is at the line of identity between false and true positives and provides no discrimination. Most agree that a clinically useful assay should have an area under the curve of 0.85 or higher.

Assay Feasibility

Most laboratory managers and directors review assay feasibility before launching complex validation, efficacy, and quality control initiatives. Feasibility study includes a review of assay throughput (number of assays per unit time), cost per test, turnaround time, and the technical skill required to perform the assay. To select a new instrument, the manager reviews issues of operator safety, footprint, overhead, compatibility with laboratory utilities and information system, the need for middleware, frequency and duration of breakdowns, and distributor support and service.

Laboratory Staff Competence

Staff integrity and professional staff competence are the keys to assay reliability. In the United States, 12 states have medical laboratory personnel licensure laws. In these states, only licensed laboratory professionals may be employed in medical center or reference laboratories. In nonlicensure states conscientious laboratory directors employ only nationally certified professionals. Certification is available from the American Society for Clinical Pathology Board of Certification in Chicago, Illinois. Studies of outcomes and laboratory errors demonstrate that laboratories that employ only licensed or certified professionals produce the most reliable assay results.22,23

Competent laboratory staff members continuously watch for and document errors by inspecting the results of internal validation and quality control programs and external quality assessment. Error is inevitable, and such incidents should be used for quality improvement and instruction. When error is associated with liability, the opportunity for improvement is often lost to obfuscation. The analysis of error without blame may be consistently practiced in an effort to improve the quality of laboratory service.

Proficiency Systems

Laboratory managers and directors assess and document professional staff skills using proficiency systems. The hematology laboratory manager may, for instance, maintain a collection of normal and abnormal blood films, case studies, or laboratory assay reports that staff are required to examine at regular intervals. Personnel who fail to reproduce the target values on examination of the blood film are provided remedial instruction. The proficiency set may also be used to assess applicants for laboratory positions. Proficiency testing systems are available from external quality assessment agencies, and proficiency reports are made accessible to laboratory assessors.

Continuing Education

The American Society for Clinical Pathology Board of Certification and state medical licensure boards require professional personnel to participate in and document continuing education for periodic recertification or relicensure. Continuing education is delivered in the form of journal articles, case studies, distance learning seminars, and professional meetings. Medical centers offer periodic internal continuing education opportunities (in-service education) in the form of grand rounds, lectures, seminars, and participative educational events. Presentation and discussion of local cases is particularly effective. Continuing education maintains the critical skills of laboratory scientists and provides opportunities to learn about new clinical and technical approaches. The Colorado Association for Continuing Medical Laboratory Education (http://www.cacmle.org), the American Society for Clinical Laboratory Science (http://www.ascls.org), the American Society for Clinical Pathology (http://www.ascp.org), the American Society of Hematology (http://www.hematology.org), the National Hemophilia Foundation (http://www.hemophilia.org), CLOT-ED (http://www.clot-ed.com), and the Fritsma Factor (http://www.fritsmafactor.com) are examples of scores of organizations that direct their activities toward quality continuing education in hematology and hemostasis.

The medical laboratory science profession stratifies professional staff responsibilities by educational preparation. In the United States, professional levels are defined as the associate (2-year) degree level, or medical laboratory technician; bachelor (4-year) degree level, or medical laboratory scientist; and the level of advanced degree, masters degree, or doctorate in clinical laboratory science and related sciences. Many colleges and universities offer articulation programs that enable professional personnel to advance their education and responsibility levels. Several of these institutions—for example, the University of Cincinnati, University of North Dakota, Weber State University, and University of Medicine and Dentistry in New Jersey—provide distance learning opportunities. Enlightened employers encourage personnel to participate in advanced educational programs, and many provide resources for this purpose. Education contributes to quality laboratory services.

Quality Assurance Plan: Preanalytical and Postanalytical

In addition to keeping analytical quality control records, U.S. agencies require laboratory directors to maintain records of preanalytical and postanalytical quality assurance and quality improvement efforts.²⁴ Although not exhaustive, Table 5-8 lists and characterizes a number of examples of preanalytical quality efforts, and Table 5-9 provides a review of postanalytical components. All quality assurance plans include objectives, sources of authority, scope of services, an activity calendar, corrective action, periodic evaluation, standard protocol, personnel involvement, and methods of communication.²⁵

Stat orders and timeliness Do turnaround time expectations match clinical necessity and ensure that stat orders are reserved for medical emergencies? Specimen collection Specimen transport Are specimens delivered intact, sealed, and in a timely manner? Are they maintained at the correct temperature? Specimen management Are specimens centrifuged correctly? Are tests begun within specified time frames? Are samples stored properly? Are coagulation specimens platelet-poor when specified?

Agencies that Address Hematology and Hemostasis Quality

The following are agencies that are concerned with quality assurance in hematology and hemostasis laboratory testing:

Summary

Now that you have completed this chapter, go back and read again the case study at the beginning and respond to the questions presented.

Review Questions