Concept validity

Concept validity is the weakest form of validity. It amounts to no more than the procedure in question sounds like a good idea. In formal terms it is that the procedure appears in theory to have a reasonable anatomical or physiological basis.

Diagnostic blocks have concept validity on the grounds that it sounds reasonable that if a structure is a source of pain, anesthetizing it will relieve that pain. Whether or not the block actually anesthetizes that structure, and whether or not the pain is genuinely relieved are not at issue in this context. Those properties of the test are covered by more senior subtypes of validity. Concept validity is limited to the abstract, i.e. theoretical, basis of the test.

Content validity

Content validity amounts to no more than the test being accurately defined. It ensures that one person’s version of the test is the same as another person’s. If this is not the case, what one practitioner performs as a particular diagnostic block may not be what another practitioner performs when they report using the procedure by the same name.

Spelling out the definition of a procedure in detail secures content validity. An example might be that fluoroscopic guidance should be used. A fluoroscopically guided block will not necessarily be the same as a ‘blind’ procedure.

Content validity does not render the procedure itself valid, but it ensures that the name of the procedure is used consistently to mean the same thing. A procedure will lack validity if a practitioner fails to observe all the details contained in the definition. Under those conditions, the procedure executed may fail to achieve correctly what it is supposed to achieve or what others achieve when the procedure is correctly performed according to its strict definition.

Face validity

For a diagnostic block to have face validity it must be shown that the block actually does what it is supposed to do in an anatomical or physiological sense. If a particular structure is said to be the target, it must be shown that the structure is anesthetized. Too often, some diagnostic blocks are assumed to have face validity simply because their name implies that they have.

Face validity can be tested and established in either of two ways. A study might be conducted to show, as a general rule, that the block always reaches its target. If other studies replicate those results, practitioners can be assured that if they perform the block correctly they, too, can expect that it will retain face validity. The alternative is to test for face validity in each in every case. This might be done anatomically or physiologically. Fluoroscopy and a test dose of contrast medium, prior to a block, can be used to show that the injectate goes correctly to the intended target. This establishes face validity anatomically. A physiological approach might be used if the target has some detectable and testable function other than pain. Thus, a sympathetic block can be tested by change in temperature or change in blood flow.

Face validity, however, is not limited to showing that the intended target is accurately reached. It also requires showing that other, potentially confounding, targets are not also affected. The objective of face validity is to show that the intended target is selectively, or discretely, anesthetized. Flooding everything in the vicinity of the target with local anesthetic does not secure face validity, for under those conditions, the operator cannot tell if the effects of the block are due specifically to the intended target having been anesthetized or to some other, unintended structure having been blocked.

Construct validity

Construct validity is perhaps the most critical of the subtypes of validity. It establishes if the test actually achieves what it is supposed to achieve. It measures the extent to which a test correctly distinguishes the presence but also the absence of the condition that the test is supposed to detect. In colloquial terms, construct validity measures if the test actually works or not, and how well it works.

In the past, it was the habit of medicine to believe that all test results were correct and true; that if a test result was positive, the condition truly was present; and that if the test result was negative, the condition was definitely not present. This tradition was been refuted and supplanted.

There is a science that involves testing tests. It involves comparing, in the same sample of patients, the results of a test with unknown validity with the results of some other test whose validity is beyond question. That latter test is known as the criterion standard, formerly known as the ‘gold’ standard.

In reality, no test is perfect; and no criterion standard is absolute. The working definition of a criterion standard is that it is a test about whose results there is substantially less dispute than the test undergoing scrutiny. Examples of a criterion standard might include imaging findings, operative findings, or what a pathologist finds at postmortem. In practice, the criterion standard is usually a test that allows a more direct detection of the condition in question than the test under scrutiny, and which is less subject to errors of observation.

When a test is compared with a criterion standard, the results can be expressed as a contingency table (Table 16.1). Such a table shows the number of patients who have the condition according to the criterion standard, and how many do not; and in how many of each category the test in question was positive or negative. Four cells emerge. The ‘a’ cell is the number of patients in whom the condition is present and in whom the results of the test are positive. These are patients with true-positive responses. The ‘b’ cell contains those patients who do not have the condition but in whom the test was nevertheless positive. These responses are false-positive. The ‘c’ cell represents those patients who have the condition but the test is negative. These responses are false-negative, for the test failed to detect the condition when it should have done so. The ‘d’ cell represents those patients who do not have the condition and in whom the test is negative. The test correctly identified these patients as not having the condition, and their responses are true-negative.

From such a table, several descriptive statistics can be derived, which can be used to quantify the virtues of a diagnostic test, or the lack thereof. Paramount amongst these are the sensitivity and the specificity of the test.

Sensitivity is the extent to which the test correctly detects the condition that the test is supposed to detect. Conceptually, this is read down the first column of the figure. Numerically, sensitivity is the ratio between ‘a’ and ‘a+c’, for ‘a’ is the number of patients known to have the condition in whom the test was positive, while ‘a+c’ is the total number of patients who had the condition. Sensitivity is also known as the true-positive rate, for it describes the proportion of cases who should have been positive that the test actually did find, correctly, as positive.

Specificity is the extent to which the test correctly detects the absence of the condition. Conceptually it is read up the second column of the figure. Numerically, specificity is the ratio between ‘d’ and ‘b+d’, for ‘d’ is the number of patients known not to have the condition in whom the test was negative, while ‘b+d’ is the total number of patients who did not have the condition. Specificity is also known as the true-negative rate, for it describes the proportion of cases who should has been negative that the test actually did find, correctly, as negative.

A companion statistic is the false-positive rate. This is the proportion of cases who did not have the condition but in whom the test was, incorrectly, positive. Numerically it is the ration of ‘b’ to ‘b+d’. It is also the complement of the specificity, i.e.:

Failure to recognize both the occurrence and the prevalence of false-positive responses has been one of the major transgressions of medicine in the past. It is both false and illusory to assume that every test result that is positive is correctly positive. A test can be positive for reasons other than the sought-for condition being present. Unless the prevalence of false-positive results is known, the validity of the test remains in question, for an investigator cannot otherwise tell if a positive result is true-positive or false-positive.

The significance of false-positive responses can be realized by analyzing the contingency table. The total number of positive responses to the test is the number of true-positive cases (‘a’) and the number of false-positive cases (‘b’). The confidence that an investigator can have, that a given positive response is true-positive, is determined by the ratio of ‘a’ to ‘b’, or the ratio of ‘a’ to ‘a+b’. The greater the value of ‘b’, the less confidence an investigator can have that a given positive response is true-positive. In other words, false-positive responses compromise diagnostic confidence.