The views expressed in this article are solely the views of the author and do not necessarily reflect the views of The WAGER, its sponsors, or affiliated organizations.
For past editorials, click here.
Letters to the Editors
Recently, The WAGER published an editorial focusing on the ethics of accepting industry funding for scientific research. This editorial prompted interesting responses that were published with the original editorial. This week we are publishing two responses from WAGER readers to the original editorial exchange as well an editorial comment.
Dear Editor:
I was interested in a recent issue concerning ethical dilemmas that may arise in accepting industry funding for gambling research. My views were similar to those of Charles Wellford, in that there was an unfortunate superficiality in the treatment of the issue. I was amazed at the underlying assumption of Howard J Shaffer that ‘advocacy’ is somehow opposed to objectivity in research (‘objectivity’ in research being a rather archaic concept anyway). This exposes a fundamental philosophical confusion. Many respected scientists have discovered grave danger to humanity in one way or another e.g., through environmental pollution, bombs, tobacco, etc. That they have felt an imperative to advocate for the community in the face of these dangers cannot be said to bring the integrity of their research into question (the concept of ‘integrity’ being much more relevant than objectivity). Using Shaffer’s criteria any researcher highlighting the dangers to the community of certain types of gambling may be suspect as an ‘advocate’. My other comment is in relation to the content of The WAGER – a topic canvassed recently, I notice. I must say that the strictures of the evident positivist framework mitigate against articles saying anything much that is useful or elucidating (which can be rather useful for industry advocates who would rather not look at the ‘big picture’ wherein the research findings become meaningful). As a researcher with a behavioral science background as well as training and experience in social, qualitative research, I found it best, in a newsletter I was involved in, to ‘look behind’ the research and give readers the critical tools by which to make sense of the research. We also had an approach of placing research in the context of the world under study. After all, researchers do not live on some rarified level apart from the social world in which the research occurs. We received extremely positive feedback to this approach and it evidently helped inform debates about gambling research in Victoria, which demonstrate a high level of awareness and sophistication. In contrast, I find that The WAGER ‘drip feeds’ isolated pieces of information that have little meaning on their own. As a result, I believe that The WAGER is unintentionally patronizing and gives little respect to the intelligence of the reader. At the same time the research is not so likely to ‘scare the horses’ and jeopardize lucrative industry patronage, which may be a happy ‘synergy’ rather than a contrivance. Thank you for the opportunity to give this feedback.
Sincerely,
-Jennifer Borrell
Dear Editor:
I am rather astounded at the charming but dangerous innocence displayed by some of the responses to the useful imaginary dialogue brought to us by the colleagues from New Zealand. One often wonders where the colleagues Wellford, Hesterman and especially Shaffer and LaPlante have been for the last quarter century and managed to be so aloof to the ethics debates and the debates about epistemology and such in the ‘free’ (and not the ‘free trade’) world. Shaffer and LaPlante’s third paragraph about advocacy really cuts the cake; their rather irresponsibly objectivist interpretation – which itself is clearly ‘advocating’ for the interests of that most improbable group of ‘objectivist peer reviewers’, who hover above the real world but are not yet entirely ascended into the ephemeral heavens of translucent interest-less-ness – should really not have been written. On the other hand, it does explain a lot… and we should probably be grateful for it having been written….? Why did Einstein, Bohm and so many of the others writing in the context of that hardest of hard areas of scientific endeavour – physics! – have to go at length to reject this kind of objectivity for THEIR sciences (the models of scientificity!!!) and have SOCIAL and HUMAN sciences continue to relentlessly pursue it…? The canons of ‘objective’ science and the objective cannons of the powers for whom this ‘objective’ and ‘value neutral’ science has been and continues to be bought have been so close for – yes – centuries, that even the rhetoric Shaffer and LaPlante use should somehow have become illegitimate… or just laughable. But I should stop here, lest my further words will be interpreted as ‘advocacy’ (rather than social responsibility and necessary precaution!)
-Jacques Boulet
On Science, Integrity, Objectivity, Values and Bias: Charming Innocence or Rigor?
It is not surprising that a dialogue about science, objectivity, values, funding and the potential for investigative bias would stimulate strong feelings about the best way to maintain scientific integrity and advance the field of gambling studies. There are three primary types of scientists: realists, instrumentalists and relativists (Casti, 1989). We agree with Boulet and Borrell that the scientific realist is an endangered species. Others have echoed this sentiment by suggesting that those who consider “…that there is an objective reality ‘out there’ independent of ourselves…” are diminishing in numbers despite the observation that “… this is the position to which most working scientists subscribe” (Casti, 1989, p. 24). We also share with Boulet and Borrell the belief that integrity of research is essential to advancing science. Despite these common views, we also disagree with Boulet and Borrell: these differences of opinion reflect dissimilar perspectives on “objectivity” and “advocacy.” In this comment, we will (1) expand upon our previous commentary that focused on advocacy and dishonesty as primary risks to scientific integrity, (2) note two different types of objectivity, and (3) briefly revisit the issue that started this dialogue by considering the relationships among values and science.
Advocacy and Bias
Jennifer Borrell was concerned that our position permitted “…any researcher highlighting the dangers to the community of certain types of gambling [to become] suspect as an ‘advocate’,” and “…that ‘advocacy’ is somehow opposed to objectivity in research (‘objectivity’ in research being a rather archaic concept anyway).” Similarly, Jacques Boulet closed his letter by saying, “I should stop here, lest my further words will be interpreted as ‘advocacy’ (rather than social responsibility and necessary precaution!).” Mistakenly, Borrell and Boulet worry that we oppose all advocacy. Advocacy has a time and place; however, that time and place is not in the midst of research, within the scientific method. Attempting to integrate advocacy into the scientific method corrupts science. For example, common sense, logic, and social responsibility often tempt us into believing that scientific tests of our hypotheses are not needed – but nothing can be further from the truth.
Advocacy can impact science during three decisive times: before, during, and after. Borrell and Boulet do not consider advocacy to be a threat to science and make no mention about the appropriateness of advocacy during any of these times. In our previous editorial comment, our concern was primarily with advocacy during the scientific process: advocacy that specifically impacts study design, study implementation, and the interpretation of study results. Prescience advocacy can be beneficial. Pre-science advocacy can prevent blatantly unethical research studies (e.g., certain types of experiments conducted on human beings). Post-science advocacy also can be beneficial, though the appropriate timing of such advocacy is often difficult to determine. Post-science advocacy must wait until sufficient evidence has been amassed for science to have an amply developed opinion. Until the evidence is “mature,” the potential for conflicts of interest is high. The most difficult part of post-science advocacy is deciding when we know enough to warrant taking off our scientist hat and putting on our citizen hat.
A close reading of our last editorial comment reveals that our concern was not simply with advocacy, but also with (1) the biases it naturally introduces into the scientific process and (2) the temptation it naturally evokes to mislead and inaccurately identify scientific evidence. For example, consider the true story of a professor who was taken by a pet theory and was a vocal advocate of its accuracy. One day in a lab meeting when a graduate student was presenting results the professor proudly exclaimed, “That’s perfect, it fits my theory exactly!” The graduate student then realized that she had exchanged two integral averages and when the professor saw the corrected pattern of data she exclaimed, “Even better! That’s perfect for my theory!” This true story might be an extreme case; nevertheless, it shows the problem of using advocacy colored glasses to view scientific data.
Everyone is an advocate for something, even if that means advocating against advocating. But, this logical conundrum misses the point entirely. As we argued in our last editorial comments and as a careful study of the history of science shows, over time scientists consistently challenge conventional wisdom and advocate for something different: a new theory, model or view. This kind of alternative position-taking is not the advocacy that we criticized in our editorial comment (Shaffer & LaPlante, 2003). Our concern is with the one-sided advocacy that always knows best, has all the answers, argues only from limited evidence, and attempts to influence the scientific process: in other words, the active support of a predefined and explicit agenda. Such misplaced advocacy is not difficult to come by: all alcohol is bad; all gambling is dangerous; industries cause bad things to happen; people are weak in the face of temptation, etc. This kind of advocacy can corrupt science just as easily as countertransference can corrupt psychotherapy. Advocacy engenders the value that the ends always justify the means – that an impure scientific process is irrelevant so long as the findings support the stated agenda.
Contrary to Boulet’s waxing about the “ephemeral heavens of translucent interest-less-ness…” scientists have considerable interest in their work and frequently take positions. However, scientists must continually challenge themselves and their positions with the best available evidence. Many of the best scientists work tirelessly to find exceptions to their rules. To reiterate, contrary to Boulet and Borrell’s suggestion that we believe scientists should be without values, we simply advocate that scientists play by the rules of science. This is what we meant by scientific objectivity. Scientific objectivity via the scientific method is intended to help scientists manage their personal values.
Like advocacy, theories and associated models also can shift our view of the evidence (Bakan, 1967; Cohen, 1985; Kuhn, 1970). Scientists who subscribe to advocacy positions or specific theoretical frames might be biased in their interpretation of data or blinded entirely to evidentiary details. Today, for example, when the sun rises in the east and sets in the west, we recognize the movement of the sun across the sky as evidence for the rotation of the earth around the sun. Many years ago, the very same evidence supported the view that the sun was rotating around the earth. When our perspectives change, the evidence can change its meaning. Scientists recognize the recursive loop that assures evidence will inform theory and theory will evolve in light of shifting evidence. Advocates, however, are not required to play by these rules. Consequently, advocates often believe that they know best and are quick to direct others to behave in certain ways—despite the evidence.
Recently, Ondersma (2002) nicely rejected the very architecture of the Boulet and Borrell criticisms when he described the dilemma faced by scientists: “Researchers are individuals and as such may have strongly held values, but they are required to seek objectivity…. the extent to which researchers stay within their findings and strive for objectivity in their presentation is a crucial measure of ethicality” (pp. 141-142). “Nowhere is this truer than when research examines controversial topics…. The conflation of advocacy and science is a clear breach of ethical principles…” (p. 142). Science has a great responsibility that requires “…the vigorous defense of scientific freedom. It also requires objectivity, protection against misuse, and openness to valid criticism” (p. 142).
Technical and Epistemological Objectivity
In response to our statement that “ironically, advocacy represents a more harmful threat to science … than academic dishonesty” and that “… scientific objectivity should be at the forefront of all scientific agendas” (Shaffer & LaPlante, 2003), Jacques Boulet was surprised by our “…charming but dangerous innocence…” He criticized that we offered a “rather irresponsibly objectivist interpretation—which itself is clearly ‘advocating’ for the interests of that most improbable group of ‘objectivist peer reviewers,’ who hover above the real world but are not yet entirely ascended into the ephemeral heavens of translucent interest-less-ness – should really not have been written.” Jennifer Borrell similarly suggested that we argued for objectivity in the realist sense and noted that objectivity is an “archaic concept.” Borrell and Boulet’s equating of scientific objectivity with objectivism reveals a flawed fundamental assumption about our beliefs: namely that we believe the avoidance of advocacy allows science to find “true” reality. Rather, as we will discuss below, we hold that scientific objectivity is a technical objectivity reflected in the scientific method and that truth evolves over time.
With less presumption and just a bit more evidenced based research, Boulet and Borrell would have discovered that Shaffer is not a scientific realist as they assumed, but rather a constructivist as evinced by many of his writings (Shaffer, 1986, 1987, 1991; Shaffer, Hall, & Vander Bilt, 1997; Shaffer & Robbins, 1991, 1995). For relativists or constructivists, "…truth is no longer a relationship between a theory and an independent reality, but rather depends at least in part on something like the social perspective of the person holding the theory. Thus for a relativist, as one passes from age to age, or from society to society, or from theory to theory, what’s true changes. In this view it’s not what is taken to be true that changes: au contraire, what changes is literally truth itself" (Casti, 1989, pp. 25-26). By failing to gather sufficient evidence, Boulet and Borrell’s assumption about our philosophy of science was biased. This circumstance precisely illustrates some of the concerns we have about the risks and hazards of advocacy.
Objective science progresses slowly using a set of rules: these principles guide the generation of knowledge (e.g., Carnap, 1966). Social and personal values direct the dissemination and application of knowledge (i.e., post-science advocacy). As we noted earlier, social and personal values might influence pre-science advocacy: whether scientists even undertake the task of generating knowledge in a certain area (e.g., cloning). However, it is very important to note that the links between generated knowledge and the consequences of that knowledge can be obscure from the start (e.g., cell biology and cloning). Technical objectivity (i.e., the use of the scientific method), as opposed to epistemological objectivity (i.e., realism), requires that scientists distinguish their opinions and subjective states from the evidence garnered by systematic research before reaching a conclusion (e.g., logical positivism). As a result, sometimes science takes more time than advocates think a social cause can endure. This can be problematic because the scientific method does not restrain advocates as it does scientists. For example, advocates can formulate an objective to discredit all industry funded research, regardless of its scientific merits or human utility. Certainly, no one would want to dismiss a cure for cancer simply because it derived from research funded by a pharmaceutical company.
Social advocates seek evidence to support their pre-existing positions. Scientist advocates construct research studies to support their pre-existing positions. Technically objective scientists test a broad range of positions (i.e., null hypotheses) by seeking and evaluating all the available evidence. Technically objective science examines every side of a controversy, as opposed to pursuing support for only one side of a debate. Social advocates and scientist advocates enter science with an agenda. Doing so, however, necessarily limits the value of science. For example, investigators easily can setup studies to demonstrate a point; but that does not mean the results will have external validity and generalize to a larger or different population. In the operational sense, scientific advocacy is analogous to choosing a poor study sample and attempting to generalize it to a broader population. Similarly, methodological biases can yield findings in the service of advocacy. For example, experiments can be manipulated to yield the pretense of appropriate methodology but remain biased toward an a priori position. Consider the following two rule-based examples. When you don’t want to find a difference between groups, use only a few observations and rely on the mathematics that yield larger standard errors for smaller samples. Alternatively, when you want to exaggerate a finding use a non-representative sample more likely to have (or not have) the characteristic of interest (e.g., GA members) and then extrapolate that finding to a representative segment of the population.
Funding sources are irrelevant to these processes. For example, in a recent unfunded and currently unpublished study of 99 Gamblers Anonymous (GA) members that garnered considerable media attention, investigators claimed to provide important estimates of the social costs associated with gambling. However, these investigators selectively used prevalence rates assuring inflated estimates of social cost; they failed to include other relevant rates of gambling prevalence that were readily available from the same source that provided the higher rates—perhaps because these estimates would lower their social cost estimates (Benston, 2003). Anti-gambling interests seized upon this unpublished report as scientific evidence for their pre-existing position. Of course, one could argue that this work was simply bad science; however, what incentive is there to do good science if existing research—whether scientifically sound or not—already supports your agenda?
Values and Science
In a world filled with conflicting values, the scientific method provides instructions for the careful consideration of complex and controversial problems. Although integrity drives quality research, scientists cannot always anticipate how their research will be used. This is true, in part, because useful science often extends beyond the life span and into areas of investigation that were unanticipated by the original investigator. Further, while research products can be used in devastating ways, these very same products also can be used to better the human condition. The very same nuclear energy that can sustain livable conditions or destroy the world, one day might be required to take us away from this planet to a new home in the universe. The value of such powerful discoveries remains to be determined. Current “truth” will likely evolve and mutate with more experience and advancing technological tools.
Borrell is troubled that the WAGER “’drip feeds’ isolated pieces of information that have little meaning on their own” and is “…unintentionally patronizing and gives little respect to the intelligence of the reader.” Actually, The WAGER follows in the classic philosophy of science tradition by respecting the intelligence of readers to integrate information and make-up their own minds. As Kaplan (1964) has noted, matters of public policy are extrascientific rather than antiscientific. “This is the conception of science as a neutral instrument, which can be used equally for good or for ill. Science provides only means, for ends which are to be determined outside the scientific enterprise” (p. 399-400). “The scientist …as much concerned as anyone else with the determination of ends, but not as scientist, only as citizen…” (p. 400). The WAGER provides scientific evidence for citizens to evaluate, integrate and apply. Finally, Borrell’s concern that in the WAGER “… the research is not … likely to ‘scare the horses’ and jeopardize lucrative industry patronage” simply reveals that she has a pre-determined purpose for the application of gambling related science: to ‘scare the horses’. Given her position, we wonder why she bothers with science at all. Although her opinion is genuine, it is disrespectful of science, scientists and anyone who is capable of independent thinking. We believe that, armed with knowledge, people are fully capable of making up their own mind about the utility of research.
Finally, any suggestion that scientists are valueless is sheer folly. Objectivity in science is quite different from “value-less-ness.” Scientists understand that playing by the rules of science yields technical objectivity. Scientific thinking requires a careful and dispassionate consideration of causal and consequential chains of influence; conventional wisdom often conceals our ability to see these links clearly. Scientific logic requires attempts to disprove as well as prove. When scientists hold values, these positions must be grounded; there must be justification for holding research related values. “Every value has its basis, what causes it to be taken as a value; but whether any values have grounds, and if so, what they are, is very much in dispute. The difference between basis and ground corresponds to that between belief and knowledge: one may have beliefs whether or not he has evidence for them” (Kaplan, 1964, p. 387-388). Although Borrell seems to believe that we should “scare the horses,” what are the grounds for this interest (i.e., end) in “scaring the horses”? We think that science has the job of informing the horses with the best available information (i.e., means) and whether they become scared is another matter entirely.
Galilieo likely would not have challenged the conventional wisdom of his time without subscribing to the scientific values and practices of evidence-based theory. Coupled with a willingness to consider alternative points of view—a deeply held value in science—evidence shifted his view of the world. Scientifically “grounded” values diminish advocacy since scientists readily recognize that scientific evidence gradually undoes itself and this evolution might require them to retract positions for which they previously advocated.
Ultimately, a blend of scientific method and personal integrity guide the generation of knowledge. “There is no guarantee that the true and the good go always hand in hand, but it is man’s estate to reach out to both” (Kaplan, 1964, p. 410).
Howard Shaffer & Debi LaPlante
References
Bakan, D. (1967). On method: toward a reconstruction of psychological investigation (1st ed.). San Francisco: Jossey-Bass.
Benston, L. (2003, March 25). Critics: Problem gambling analysis flawed. Las Vegas Sun.
Carnap, R. (1966). Philosophical foundations of physics; an introduction to the philosophy of science. New York,: Basic Books.
Casti, J. L. (1989). Paradigms lost: images of man in the mirror of science. New York: William Morrow & Company.
Cohen, B. I. (1985). Revolution in science. Cambridge: Harvard University Press.
Kaplan, A. (1964). The conduct of inquiry: methodology for behavioral science. Scranton, PA: Chandler Publishing Company.
Kuhn, T. S. (1970). The structure of scientific revolutions (Second ed.). Chicago: University of Chicago Press.
Ondersma, S. J. (2002). Scientific freedom is not the only issue. American Psychologist, 57(2), 141-142.
Shaffer, H. J. (1986). Conceptual crises and the addictions: a philosophy of science perspective. Journal of Substance Abuse Treatment, 3, 285-296.
Shaffer, H. J. (1987). The epistemology of "addictive disease": The Lincoln-Douglas debate. Journal of Substance Abuse Treatment, 4(2), 103-113.
Shaffer, H. J. (1991). Toward an epistemology of "addictive disease". Behavioral Sciences & the Law, 9(3), 269-286.
Shaffer, H. J., Hall, M. N., & Vander Bilt, J. (1997). Estimating the prevalence of disordered gambling behavior in the United States and Canada: a meta-analysis. Boston: Presidents and Fellows of Harvard College.
Shaffer, H. J., & LaPlante, D. (2003). Editors’ comment: on scientific integrity. Retrieved March 14, 2003, from
Shaffer, H. J., & Robbins, M. (1991). Manufacturing multiple meanings of addiction: time-limited realities. Contemporary Family Therapy, 13, 387-404.
Shaffer, H. J., & Robbins, M. (1995). Psychotherapy for addictive behavior: a stage-change approach to meaning making. In A. M. Washton (Ed.), Psychotherapy and Substance Abuse: A Practitioner’s Handbook (pp. 103-123). New York: Guilford.
The WAGER is a public education project of the Division on Addictions at Harvard Medical School. It is funded, in part, by the National Center for Responsible Gaming, and the Massachusetts Council on Compulsive Gambling.