The WAGER Vol. 5(33) – Prevalence and Validity: Toward Multi-trait-Multi-method Strategies

In the last WAGER, we examined the recent prevalence estimates of disordered gambling among the British. As Table 1 reveals, using different research strategies, the British study generated estimates similar to those reported by NORC (Gerstein et al., 1999), the National Research Council (National Research Council, 1999), Harvard Medical School (Shaffer & Hall, 2000; Shaffer, Hall, & Vander Bilt, 1999), and Kallick et al.’s first na-tional estimate generated more than 20 years ago (Kallick, Suits, Dielman, & Hybels, 1979). Although the estimates are not identical-after all these are estimates-the similarity provides an increasing degree of confidence in the reliability of these different rates and the capacity for different research strategies to generate relatively consistent findings.

*NORC Classification System; April 1, 1999 version; telephone survey data only **Harvard 1997 data represents unweighted means and 95% confidence intervals based upon 120 studies; 2000 data represents Andrews’ Wave M-estimator & 95% confidence intervals based upon 160 studies ***Mean of DSM-IV and SOGS estimates for levels 2 & 3, among the population age 16 and older **** Lifetime

Taken together, these findings suggest that disordered gambling-as scientists currently understand it-is a relatively reliable phenomenon. Despite this reliability, no single prevalence estimate provides a meaningful (i.e., valid) index of various population strata. For example, every estimate in Table 1 above likely underestimates the prevalence of gambling disorders among psychiatric or substance abusing populations. Similarly, these rates of gambling problems tend to overestimate the prevalence among women and underestimate the rate among men. Although there is a modicum of reliability among estimates across studies, the question remains, are these estimates valid? Last year, Alan Leshner, the Director of the National Institute on Drug Abuse asked, "When is addiction, addiction?" (Leshner, 1999).

Leshner asked the validity question. We can add, "When is pathological gambling, pathological gambling?" For example, these studies failed to exclude the range of psychiatric disorders that can mimic or influence gambling disorders. Failing to identify the presence of these influences will yield inaccurate prevalence estimates. If the prevalence studies homogeneously fail to identify comorbid psychiatric conditions, then the estimates likely will be uniformly inflated. Often, the research literature suggests that a screening or diagnostic instrument is valid by simply citing the seminal study that created the device. However, this is not sufficient to conclude that the instrument is valid for its current application. Instrument validity always is local and current. These conceptual problems are not new.

Long ago, in a classic work, Campbell and Fiske suggested that the proper way to determine the validity of a construct is to employ multiple measurement instruments that assess multiple traits of the attribute under investigation (e.g., pathological gambling) (Campbell & Fiske, 1959). To date, like the British study and others, some investigators have done this. However, Campbell and Fiske also note that, in addition, studies need to measure attributes that should not associate with construct of interest. In general, gambling studies have not done this. When different instruments both (1) converge in the measurements of related attributes and (2) diverge in the measurement of unrelated at-tributes, then a modicum of construct validity has been achieved. Currently, existing measures of gambling disorders are highly correlated with other behaviors (e.g., risk taking, irresponsibility, fiscal mismanagement, social dysfunction, etc.) that often associate with gambling and an array of other problems. As a result, scientists may have unwittingly and inadvertently constructed instruments that are increasingly reliable, but not necessarily valid (Barron, 1998). In the next WAGER, we will examine the specific relationship between two contemporary instruments often used to identify problem gambling.

References

Barron, J. (Ed.). (1998). Making diagnosis meaningful: enhancing evaluation and treatment of psychological disorders. Washington, D.C.: American Psychological Association.

Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the mulitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.

Gerstein, D., Murphy, S., Toce, M., Hoffmann, J., Palmer, A., Johnson, R., Larison, C., Chuchro, L., Bard, A., Engelman, L., Hill, M. A., Buie, T., Volberg, R., Harwood, H., Tucker, A., Christiansen, E., &

Cummings, W. (1999). Gambling Impact and Behavior Study: Report to the National Gambling Impact Study Commission.: National Opinion Research Center.

Kallick, M., Suits, D., Dielman, T., & Hybels, J. (1979). A survey of American gambling attitudes and behavior (Research report series, Survey Research Center, Institute for Social Research). Ann Arbor, MI: University of Michigan Press.

Leshner, A. (1999, February 5). Addiction is a brain disease and it matters. Paper presented at the New Directions in Gambling Addiction Research, George Washing-ton University.

National Research Council. (1999). Pathological gambling: a critical review. Washing-ton D.C.: National Academy Press.

Shaffer, H. J., & Hall, M. N. (2000). Updating and refining meta-analytic prevalence estimates of disordered gambling behavior in the United States and Canada, submitted for publication.

Shaffer, H. J., Hall, M. N., & Vander Bilt, J. (1999). Estimating the prevalence of disordered gambling behavior in the United States and Canada: a research synthesis. American Journal of Public Health, 89, 1369-1376.