Several recent WAGERs (e.g., WAGERs 9(13) and 9(17)) noted that self-report might limit research on pathological gambling. It is not uncommon for people to report faulty data due to recall biases and the desire to make a positive impression on others: two conditions that could significantly alter results. However, few studies have empirically tested the widely-held assumption that retrospective recall of gambling behavior and outcomes yields faulty data. This week’s WAGER presents the results of a recent study by Hodgins and Makarchuk (2003) that examined the reliability of several samples of problem gamblers’ recall of their own gambling behavior, and whether gamblers’ reports corresponded with the information provided by collateral informants.
To test the reliability of patient recall of gambling behaviors, Hodgins and Makarchuk (2003) recruited 35 subjects from a larger study on relapse (Hodgins, el-Guebaly, & Armstrong, 2001) and interviewed them two to three weeks after their 12-month treatment follow-up interviews. All participants scored 5 or greater on the South Oaks Gambling Screen (SOGS, Lesieur & Blume, 1987) at the beginning of the relapse study. They had gambled at least once during the four weeks prior to the start of treatment and consequently were considered to be pathological gamblers. Two to three weeks after the 12month follow-up interview, an independent interviewer, blind to the results of the 12-month follow-up, re-questioned subjects about their gambling behavior during the final six months; the Timeline Follow-Back (TMFB) method of assisting recall (Sobell & Sobell, 1996) was used in both interviews. Using intraclass correlation coefficients, the agreement between subjects’ 12-month follow-up and re-test responses for both days and dollars spent gambling (see Table 1), was reasonably high. Though the authors found that subjects reported significantly more gambling days at the second interview, they concluded that the results indicate that the TLFB resulted in reliable retrospective self-reports of gambling behavior.
Table 1: Test-Retest Means and Reliability Coefficients for Days Gambled and Dollars Spent
Note: N = 35. Mean retest interval = 22 days (range: 11-38). Z = Wilcoxon signed ranks test; ICC = intraclass correlation coefficient. *p< .05 **p< .01
To test the validity of patient recall of gambling behaviors, Hodgins and Makarchuk (2003) used two distinct sample groups from the original study (Hodgins et al., 2001). This analysis tested the agreement of subject responses with information provided by collateral informants, including spouses, family members, friends, and roommates. In sample one, 58 of the 102 participants provided a collateral informant who was successfully contacted, while in sample two 66 of the 101 participants provided a collateral informant who was successfully contacted; the authors contacted these collateral informants following each subjects’ three-month follow-up interview. Interviewers asked informants to provide information about subjects’ gambling history and days spent gambling over the previous three months. Comparisons with subjects’ three-month follow-up interview revealed high correlations for gambling days during the past three months (ICC = .73) and length of problem (ICC = .93). Agreement about the gamblers’ marital status was also high (k = .76, 82% agreement); agreement for treatment involvement was slightly lower (k = .43, 72% agreement). Non-spouses generally showed higher agreement with subjects than did spouses; however, agreement was higher than the authors had predicted across all variables.
The results from both the reliability and concordance segments of this study suggest that self-reported gambling behaviors are generally accurate and consistent over time. However, there are several limitations to this study. For example, all participants in this study were treatment seekers. Although this population generally agreed with collaterals and provided accurate information over time, it is unknown whether evaluation of non-treatment-seeking gamblers would yield similar results. In addition, the samples used in this study were small and might not have been representative of the general problem gambling population or the general problem gambling treatment seeking population; future studies should incorporate larger randomly selected samples to maximize the generalizability of the results to the wider population. Furthermore, though participants in both samples used to test concordance were classified as pathological gamblers based on their South Oaks Gambling Screen scores, only participants in sample 2 were formally diagnosed using the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV). Different inclusion criteria also could be a limiting factor. For example, consolidating two samples defined by different standards as one might influence the results and mislead the data interpretation. Finally, because this study did not monitor actual gambling activity (i.e., only collected self-reported data), it is unknown whether the behaviors recalled by subjects and corroborated by collaterals were accurate representations of actual gambling behaviors. Although monitoring actual gambling activity might prove to be difficult, the next step in reliability and validity research on self-report requires this valuable data to clarify stakeholders about the precision of subject recall.
Despite these limitations and future research needs, Hodgins and Makarchuk (2003) provide an informative view of the reliability and validity of subject self-report. Although additional studies are necessary to support these results, this study suggests that data acquired through retrospective recall is perhaps more dependable than is generally assumed. As a result, researchers and public policy makers can view many existing studies in a new light and approach future research with renewed confidence in self-reported data.
Comments on this article can be addressed to Tony Donato or Siri Odegaard.
References
Hodgins, D. C., el-Guebaly, N., & Armstrong, S. (2001). Critical dimensions of relapse in pathological gambling. Calgary, Alberta, Canada: University of Calgary.
Hodgins, D. C., & Makarchuk, K. (2003). Trusting problem gamblers: reliability and validity of self-reported gambling behavior. Psychology of Addictive Behaviors, 17(3), 244
248.
Lesieur, H. R., & Blume, S. B. (1987). The South Oaks Gambling Screen (SOGS): A new instrument for the identification of pathological gamblers. American Journal of Psychiatry, 144(9), 1184-1188.
Sobell, L. C., & Sobell, M. B. (1996). Timeline Follow-Back user’s guide: A calendar method assessing alcohol and drug abuse. Toronto, Ontario, Canada: Addiction Research Foundation.