The Sober Truth

For almost 80 years, AA and its fellow 12-step programs have dominated addiction treatment in this country.  I recently reviewed the scientific literature on the success of AA programs for my new book, The Sober Truth: Debunking the Bad Science Behind 12-Step Programs and the Rehabilitation Industry.  The results were stark: 12-step programs have a 5-10% success rate, making them statistically one of the least effective treatments in medicine. In fact this rate is less than several estimates of the spontaneous remission rate for alcoholism, meaning that roughly the same number of people get better on their own as do in AA.

In 2006, a massive Cochrane Review of all studies on 12-step programs in the past 40 years showed that there was simply no scientific evidence for their effectiveness.  In The Sober Truth, we looked again at these studies, but also at some of the research that failed to meet the Cochrane Collaboration’s strict guidelines for experimental design, because many of these studies are cited frequently by AA’s advocates. What we found was deeply concerning.  Every study that concluded it had found evidence for 12-step effectiveness included at least one disqualifying statistical or logical error, and many had several.  Findings were regularly based on correlations, not causations, and practically none of the studies looked at data beyond one year, a remarkable fact for a lifelong condition.  Very few studies were controlled or randomized, which is the standard in scientific inquiry.  And many suffered with selection bias: subjects chosen for a study of 12-step effectiveness had previously been exposed to, or opted into, 12-step treatment.

The most serious problem of all was a widespread tendency to discard data that did not fit the authors’ conclusions.  Many studies only analyzed the patients who chose to remain in the study, despite universally acknowledging that the vast majority of patients dropped out of every study – and that these were the people with the worst outcomes.  (The huge dropout rate, incidentally, is consistent with AA's own assessment that approximately 75% of those who attend leave the program by the one-year mark.)  Drawing conclusions and making recommendations about the suitability of a treatment based upon the small minority that remains is unacceptable.

Besides the tautological nature of measuring only those who succeed, these studies were guilty of a related error: ignoring the phenomenon of compliance bias.  It has been widely demonstrated that in any group intervention, those who comply with the intervention (be it a pill, exercise, or attending 12-step meetings) will always do better than those who do not.  In one famous example, people who took placebos were shown to live longer than those who forgot or refused to take them.  This bizarre finding only makes sense when you consider that "compliers" are simply different from non-compliers.  In this example, their longer lives were a product not of the placebo, but of the fact that they were the kind of people who internalized and executed health advice faithfully, including all kinds of recommendations about healthy eating and exercise.  The 12-step studies likewise extrapolate from the group of compliers who stayed in the studies.  Without randomizing people to either stay in or stay out of AA, we can never be sure whether we’re looking at a true cross-section of addicts.

To give one example (among many), we looked at a study in which researchers followed patients for 16 years and concluded, "The results support the benefit of extended engagement in AA" (Moos, 2006).  It was an impressive length of time to research and its conclusion seemed clear.  But a closer look revealed virtually the opposite result.  What began as a study of 628 people ended up losing 83% of its members; the authors based their 16-year conclusions on the 107 people who remained.  Although the authors acknowledged that the 83% who dropped out fared the worst, these people statistically disappeared.  The data were also gathered via a self-reporting system without independent verification, and were gathered for brief, separate windows of time amounting to just 12% of the total time period of 16 years.  There were no controls and no randomization, and conclusions were based on correlations between attendance and results, a connection which the authors themselves acknowledged did not demonstrate causality.  Notably, the authors acknowledged most of these problems themselves. Yet, they still found that the study's results supported extended engagement in AA, a conclusion not justified by the study's data or methodology.  This conclusion was also circular, because saying that the people who stayed longest did best overlooks the fact that those doing best stayed longest.  This is not a small point, because it is precisely those who self-select into AA for reasons of their own who do well, which means that it is a mistake to try to refer into AA people who do not find it helpful.

Now is the time for a thorough re-evaluation of the prominent place we give to12-step treatment.  We need to institute careful screening before sending anyone to independent AA meetings or to the rehabilitation centers founded on the 12 steps.  We need to assess whether the people in our care represent members of the small group who will gain from it, or whether they are more likely to succeed through a professional evaluation of the factors leading to their compulsive behavior.

AA’s current privileged position wouldn’t be an especially urgent problem if the treatment were simply harmless.  But often that is not the case: many people find their experiences and disappointments within AA to be deeply painful, leading to even greater feelings of failure after failing to thrive in an organization that is presumed to be for everyone.  And of course when 90% of patients are receiving ineffectual care, they are wasting valuable time that could be spent pursuing more helpful treatment. 

Our dogmatic emphasis on a single treatment approach is a grave public health issue.  Considering the harm done to the vast majority of referrals who cannot make use of this approach, it is time to stop blindly referring to 12-step programs and to open ourselves to other alternatives.

—

Lance Dodes, MD
Training and Supervising Analyst Emeritus, Boston Psychoanalytic Society and Institute
Former Asst. Clinical Professor of Psychiatry, Harvard Medical School (retired)