In the last WAGER, we noted that prevalence estimates across major studies were relatively reliable, suggesting that the problem gambling construct under investigation in these studies was similar. However, in spite of this reliability, we also raised the important question of whether this construct was actually what it has been thought to be. While most commonly used, the concordance among various measures is only one index of validity (i.e., convergent validity). Although rarely assessed, screening instruments also should provide very different estimates when they measure conceptually different constructs (i.e., discriminant or divergent validity). In this issue of the WAGER, we will examine whether the cases that comprise a prevalence estimate generated from two different screening instruments correspond or conflict.
Over the years, investigators have criticized the South Oaks Gambling Screen (SOGS) (Lesieur & Blume, 1987) for yielding inflated estimates of disordered gambling prevalence. For example, in the recent British study (Sproston, Erens, & Orford, 2000), the SOGS yielded past year prevalence rates (.8%) that were higher than those derived from a DSM-IV (.6%) based gambling screen. Shaffer, Hall and Vander Bilt noted a similar difference in the prevalence research on gambling disorders throughout the United States and Canada (Shaffer, Hall, & Vander Bilt, 1997). In spite of these estimate differences, investigators typically have not addressed how the cases identified by these two instruments correspond.
By comparing respondents that were identified by different screens, scientists can assess the relative sensitivity and specificity of each instrument. The sensitivity of an instrument is its capacity to identify an attribute of interest when it is present (e.g., identifying a problem gambler when they are a problem gambler). The specificity of a measurement device is its capacity to not identify the attribute of interest when it is absent (e.g., not identifying a problem gambler when they are not a problem gambler).
Sproston et al. (2000) conducted a cross-tabulation of the SOGS and DSM-IV screens used in their study. They reported that 99% of the population sample were classified as "non-problem gamblers" on both instruments. Then, after removing these respondents from consideration, Sproston et al. examined the remaining people who were identified as a problem gambler by either screen. These percentages are summarized in the table below.
The authors interpret their findings as follows: "almost two thirds (64%) of people who were classified as problem gamblers by the DSM-IV, were also problem gamblers according to the SOGS" (p. 52). Since the SOGS provided a higher prevalence estimate than the DSM-IV screen, it can be expected that a higher percentage of these people would not be identified (56%) as problem gamblers according to DSM-IV. "Conversely, over a third (36%) of people who were classified as problem gamblers according to the DSM-IV, were not classified as problem gamblers by the SOGS. This suggests that it is not simply the case that the SOGS has a lower sensitivity for measuring problem gambling than the DSM-IV" (p. 52).
The authors concluded that both instruments reflect false negatives and false positives. In addition, the SOGS fails to identify some cases that are identified by the DSM-IV screen in spite of its tendency to yield higher estimates. Taken together, these findings help us to understand that prevalence estimates aggregate false negatives and false positives to yield relatively reliable population estimates across instruments. However, this observation also reveals why screening devices are inadequate to make a diagnostic or treatment decision for individual cases. In addition, these differences raise the possibility that different screens might measure similar but different underlying constructs. This sticking point to establishing validity is not limited to the study of gambling disorders; it extends throughout the study of psychiatric disorders.
References
Lesieur, H. R., & Blume, S. B. (1987). The South Oaks gambling screen (SOGS): A new instrument for the identification of pathological gamblers. American Journal of Psychiatry, 144(9), 1184-1188.
Shaffer, H. J., Hall, M. N., & Vander Bilt, J. (1997). Estimating the prevalence of disordered gambling behavior in the United States and Canada: a meta-analysis. Boston: Presidents and Fellows of Harvard College.
Sproston, K., Erens, B., & Orford, J. (2000). Gambling behaviour in Britain: results from the British gambling prevalence survey. London: National Centre for Social Research.