National Policymakers versus Internationally Minded Researchers? Lessons from a Case Study

Editor’s Note: This piece was written by Simon Planzer, Ph.D., Lecturer at the University of St.Gallen (Switzerland) and member of the Academic Council of the International Center for Gaming Regulation at UNLV. Dr. Planzer is a member of The BASIS Editorial Board.

In Europe, numerous countries have already passed or are in the process of passing online gambling legislation. It seems obvious that new gambling jurisdictions would have an interest in learning from the legislative experiences of other countries. “Yes, they should do so – but they rarely do,” has been my summary on a recent panel discussion. This Op-Ed outlines some of the reasons why policymaking gets often caught in a “national logic”. I then switch from the policymakers to the researchers: Are at least problem gambling researchers internationally minded? I illustrate this along a case study and conclude with some key lessons that policymakers globally can draw from that study.

National policymaking

Sound and effective gambling regulation should start with the fundamental question: What are our policy goals? (Cabot, 2014). And if these goals could be conflicting: What are our priorities? Setting these national goals and priorities is the prerogative of a democracy. Yet, in a second step, would it not be wise to look abroad for international best practices as to the means of how to most effectively achieve these policy goals?

All too often, the choice of regulatory means gets caught in a very “national logic”. The most obvious reason is that politicians, more precisely parliamentarians, do not get elected by something as abstract as “international best practices”. They get elected in their district. A district that may for instance include a strong casino group. Historically grown facts and economic factors further add to a national logic. It seems difficult for legislators to think outside their own legislative box. After all, they are humans and inertia is a well-described phenomenon in behavioral sciences (Thaler & Sunstein, 2009). Amendments, even very substantial ones, normally build on the pre-existing categories and patterns stemming from the old law.

Furthermore, powerful economic stakeholders, such as domestic lotteries, may manage to impose their choices already at the stage of the draft law – prior to the actual parliamentarian work. Such national factors may lead legislators to ignore the advice of their own domestic regulatory authority, too (i.e., the authority that will apply the gambling law, supervise the gambling activities and pass sanctions where needed). Switzerland is currently passing a new gambling law and that Alpine republic is no exception: the afore-described legislative deficits could be observed in the process.

The case study: Internationally Minded Researchers?

Instantly, we are tempted to think that, fortunately, science is very different. Science is international and borderless. Indeed, it would seem very odd to contain empirical sciences within national borders. Researchers look abroad. Is this the case?

Let us revisit the findings of a case study that I performed in relation problem gambling research in Switzerland (Planzer, 2014). The study inquired two questions:

To which extent can assumptions be found in research publications that are not as such supported by international empirical evidence?

To which extent do the research publications consider the relevant peer-reviewed (English language) literature?

For that purpose, a literature review identified and analyzed in detail the publications on problem gambling written by Swiss authors: 9 papers in peer-reviewed international (English language) journals, 11 articles in German language or national journals or websites, and 13 reports ordered by government bodies or agencies. Each publication was carefully assessed in relation to the aforementioned two questions.

I was surprised by the results. Numerous publications relied on far-reaching assumptions. Similarly, it was rare that the international English-language literature was duly taken into account. I should immediately add: From many discussions with colleagues from other countries, I have no reason to believe that this is a problem only specific to one country. Notably the epidemiological data situation is rather poor in many European countries (Planzer, Gray & Shaffer, 2014) and Petry noted that many studies suffered from methodological deficits (Petry, 2005). Another important factor is that methodological deficits have been noted globally in particular regarding the alleged harmfulness of (all kind of forms of) Internet use (Byun et. al., 2009).

Peer-reviewed papers on the one hand… 

A close and differentiated look at the study results offers important insights. Some findings are actually very positive and comforting in relation to scientific standards. The papers in peer-reviewed international journals showed high quality and took regularly into account the international English-language literature relevant for the topic the authors dealt with. These publications contained the only legitimate form of “assumptions”: initial hypotheses, which were then thoroughly assessed.

This finding seems to support the view that peer-reviewed journals do bring added value to scientific research. There are inherent “institutional” factors that produce quality-enhancing effects. The editor and anonymous peer-reviewers assess the quality of the submitted manuscript. They may note speculative, unbalanced or far-reaching assumptions. The (lack of) use of the relevant international literature in particular is a classic review criterion that reviewers are asked to check by the editor. Certainly, peer-reviews do not by themselves guarantee high quality publications. But the mere awareness of the peer-review hurdle is likely to produce on the authors an ex ante effect: a carefully drafted manuscript that considers the relevant literature and empirical evidence stands better chances to be accepted by the journal. And ultimately scientists want to see their work published – and not rejected in the peer-review process.

… reports ordered by the government on the other hand

The positive results in relation to peer-reviewed publications contrasted strongly with the quality of another category: national or regional reports that were ordered by the government, i.e., governmental departments, agencies or bodies. By far, they featured the most far-reaching assumptions and rarely took the international English-language literature duly into account.

The problem with this finding is that policymakers are highly unlikely to read peer-reviewed specialized journals. But they may very well read national or regional reports ordered by governmental bodies, in full or the executive summary. The scientists’ call for science-minded policymakers is more than legitimate (Shaffer, 2014). This case study established that those reports, which policymakers are likely to read and/or are likely to inform their decision-making, might feature significant deficits.

Lessons for policymakers

What lessons can policymakers draw from this case study? More precisely, governmental bodies that order research reports on problem gambling? What quality measures can we conclude from this study?

The question is all the more important as these reports are among the few pieces of research to attract the attention of policymakers. Not least, this question also regards the careful allocation of funds: targeted and effective use of public money.

Eminence based versus evidence based research

A recurring quality problem in various countries is that reports all to often rely on the personal views of a few eminences (“Let’s ask the specialists!”). In this case study, too, a methodology could often be noted that substantially relied on individual views from practitioners, notably from the therapy and counseling sector. These opinions formed a significant part of the gathered “data” on which the conclusions were based. The limitations that come with such methodology were hardly ever mentioned. One lesson accordingly is: Policymakers should insist that conclusions in reports are not eminence based but evidence based. They must ensure that any report they order takes into account the international literature, most importantly the current state of empirical evidence. This may require a more time-consuming piece of research but should at least ensure that reports are produced that do not simply reflect personal views of practitioners that may “fly by the seat of their pants” (Shaffer, 2014).

Study design

At the stage of the grant allocation, competent reviewers should carefully assess the suggested research design. The case study established that in some cases it would have been clear from the outset that the study will not be able to answer the research questions due to the chosen study design.

Public bidding

The quality can further be improved by applying call for tenders. Instead of simply allocating research grants to (the ever same) domestic researchers, call for tenders should be published, enabling researchers globally to submit their proposals. I do not see compelling reasons why research for the attention of policymakers would need to be composed by nationals or residents from that country exclusively. Since English is widely understood in many countries, alleged language barriers may reflect a convenient excuse rather than a real concern.

Peer-reviews

Policymakers ordering reports should use peer-reviews, both at the stages of research proposals and the submission of the report.

International literature and empirical evidence

Policymakers should make it a mandatory requirement of the ordered report to consider the international, notably English language literature, which is relevant for the topic at hand. This is particularly important in relation to the current status on the empirical evidence.

References

Byun, S. Ruffini, C., Mills, J.E. et al. (2009). Internet Addiction: Metasynthesis of 1996-2006 Quantitative Research. CyberPsychology and Behavior 12(2): pp. 203-7.

Cabot, A. (2014). Public Policy and Policy Goals. In: A. Cabot & N. Pindell (Eds.), Regulating Land-Based Casinos (pp. 21-53). Las Vegas: UNLV Gaming Press.

Petry, N. (2005). Pathological Gambling: Etiology, Comorbidity, and Treatment. Washington DC: American Psychological Association.

Planzer, S. (2014). Glücksspiel-Regulierung und Glücksspiel-Sucht: Annahmen und Fakten. Die nationale Glücksspielsucht-Forschung im Lichte des internationalen empirischen Erkenntnisstands (“Gambling Regulation and Gambling Addiction: National Gambling Addiction Research in the Light of International Empirical Evidence”). Bern: Editions Weblaw. The study received financial support from a public lottery (Swisslos) and a casino group (Swiss Casinos).

Planzer, S., Gray, H. M., & Shaffer, H. (2014). Associations between National Gambling Policies and Disordered Gambling Prevalence Rates within Europe. International Journal of Law and Psychiatry 37(2): pp. 217-29.

Shaffer, H. (2014). Science, Public Policy, and Law. Considering the Case of Gambling. Foreword in: S. Planzer (2014). Empirical Views on European Gambling Law and Addiction (pp. v-ix). New York: Springer International Publishing.

Thaler, R. & Sunstein, C. (2009). Nudge: Improving Decisions About Health, Wealth, and Happiness. New York: Penguin Books.

—  Simon Planzer, Ph.D.

What do you think? Please use the comment link below to provide feedback on this article.