Extremely brief screens, such as the three-question Brief Biosocial Gambling Screen or BBGS, are an economical way for clinicians to identify patients at risk for addictive behavior. Using longer screens may allow clinicians to differentiate between different levels of risk or test for more risky behaviors. However, due to time and resource constraints, longer screens are infrequently integrated into routine care. One potential solution is to install a computerized system to automate screening. This week, STASH reviews a study that examined the reliability of a computer-based screening system (McNeely et al., 2014).
Methods
- At a primary care clinic in a large hospital in New York City, over two recruitment periods (April to August 2011, January to April 2012), research assistants (RAs) recruited a total of 146 adults between the ages of 18 and 65.1
- Researchers developed a computer-based version of the Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST).
- A total of 101 participants completed the computer-based ASSIST twice, once during a first visit, once during a second visit an average of twelve days later.
- Researchers used ASSIST scores to classify participants as either low risk or moderate/high risk for problems with each of twelve categories of substances: tobacco, alcohol, cannabis, cocaine, stimulants, inhalants, sedatives, hallucinogens, opioids, prescription opioids, prescription stimulants, and other drugs.
- Participants were also categorized as moderate/high risk for problems with drugs overall if they were moderate/high risk for any category other than tobacco or alcohol.
- For each of the categories where the moderate/high risk prevalence rate for the study population was above 20%, the researchers measured test-retest reliability using Cohen’s Kappa. Essentially, they measured the concordance between risk level based on the first time using ASSIST and risk level based on the second time using ASSIST.
Results
- Cocaine, marijuana, drugs overall, alcohol, and tobacco had high enough prevalence rates to measure test-retest reliability..
- For each of these five categories, the risk level observed during the first visit was the same as the risk level observed during the second (either both low or both moderate/high) for between 92 and 95 participants (between 91.1% and 94.1%).
- Over the five categories, Cohen’s Kappa ranged from 0.76 to 0.86 (all p-values below 0.001), meaning that the results from the second visits were consistent with the results from the first visits.
Figure. Modified bar chart denoting the number of participants who screened as low risk or moderate/high risk for problems with tobacco, alcohol, drugs overall, marijuana, and cocaine, in their first and second visits using the ACASI-ASSIST. Adapted from McNeely et al. (2014). Click image to enlarge.
Limitations
- Due to the small sample sizes and low prevalence rates in the participant population, the researchers were not able to test for reliability with respect to many of the categories of substances.
- All of the participants were recruited at a single facility. The results may not be generalizable to the greater population.
- Not all of the participants recruited completed the computer-based ASSIST twice. The study may have been biased towards participants who are more capable or are more accustomed to handling computer-based surveys.
- While the original ASSIST, administered face-to-face by another person, has been validated in a study (Ali et al., 2013), the computer-based ASSIST has not. Using a computer system instead of a human might have affected the validity of the screen.
Conclusion
So long as they are reliable and valid, self-administered screeners, such as computer- or web-based systems, can save clinicians and support staff time. If patients can complete screeners such as ASSIST before meeting with their doctor, the face-to-face time saved can then be used for more in-depth conversations about any potential health problems and further assessment, when necessary. The end result of this saved time might be better doctor-patient relationships, quicker identification of emerging problems, and better care. Moving screening procedures to computers can sound like a process that would make health care cold and impersonal—almost robotic. However, giving clinicians more time to have important conversations with their patients might give them a chance to do the work that only humans can do.
– Matthew Tom (@MattTom857)
What do you think? Please use the comment link below to provide feedback on this article.
References
Ali, R., Meena, S., Eastwood, B., Richards, I., & Marsden, J. (2013). Ultra-rapid screening for substance-use disorders: The Alcohol, Smoking and Substance Involvement Screening Test (ASSIST-Lite). Drug and Alcohol Dependence, 132(1-2), 352–361. doi:10.1016/j.drugalcdep.2013.03.001
Babor, T. F., McRee, B. G., Kassebaum, P. A., Grimaldi, P. L., Ahmed, K., & Bray, J. (2007). Screening, Brief Intervention, and Referral to Treatment (SBIRT): toward a public health approach to the management of substance abuse. Substance Abuse: Official Publication of the Association for Medical Education and Research in Substance Abuse, 28(3), 7–30. doi:10.1300/J465v28n03_03
McNeely, J., Strauss, S. M., Wright, S., Rotrosen, J., Khan, R., Lee, J. D., & Gourevitch, M. N. (2014). Test-retest reliability of a self-administered Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) in primary care patients. Journal of Substance Abuse Treatment, 47(1), 93–101. doi:10.1016/j.jsat.2014.01.007
[1] To test the system on patients who might have difficulties with the computerized system, the RAs oversampled for these criteria: adults with less than a high school education, adults between 50 and 65 years old, and those who primarily spoke Spanish. The RAs also used purposeful sampling to achieve an approximately 1:1 male-to-female participant ratio.