Basic Competency Examination

NEGATIVE STUDIES AND COMMON SENSE

JOURNAL OF BIOELECTRICITY, 8(1), v–vii (1989)

EDITORIAL

During a recent trial in Florida, Phillip Cole, Chairman of the Department of Epidemiology, University of Alabama, testified regarding health risks due to chronic exposure to powerline electromagnetic fields. There are 19 studies, he said, that reported associations between disease and a surrogate for exposure, but there are 11 studies that did not report such an association. Cole doubted the validity of the positive studies because there were 80 many negative studies. Furthermore, the positive studies were “implausible” because he knew of no reason that electromagnetic fields should cause physiological changes. Consequently, Cole opined, children exposed to powerline fields would not experience increased risk of disease.

Cole’s approach dovetails nicely with the position of the national power industry. If negative studies do indeed balance out positive studies, then a convenient trap door exists to escape liability and responsibility for the disease-promoting aspects of their activities. This is one plan: Award a contract to a research-for-hire laboratory or a university professor willing to play ball, and specifically define the desired research. For example, provide that only very old rats shall be studied, thereby minimizing the possibility that specific neuroendocrine variables will exhibit a sustained response to an impressed field. Or, require that the experiment be performed in animal cages 40% smaller than normal. Since both the exposed and control animals will be significantly stressed by confinement, the possibility of observing a stressor response caused by the field will be minimized. Another strategy involves the method of data analysis: Suppose the sponsor requires that the dependent variable exhibit a dose-effect relationship with field intensity as a condition precedent to the acceptance of the occurrence of a field-induced effect. If the dependent parameter is altered (compared to the control) at all field intensities studied, but does not exhibit a linear relationship with intensity, it can be concluded that no effect was observed. Entering into a contract with an inept scientist is another method for bringing about negative studies. Ignorance then becomes a virtue for the sponsor because there is only a minimal possibility that useful information will be obtained. If anyone doubts that this overall strategy actually exists, let him explain the plethora of industry-bankrolled negative studies.

Every worthwhile scientific study is performed to test a hypothesis: The experimental hypothesis relates to the meat-and- potatoes of the study, but it is the statistical hypothesis that is formally tested. This statistical (null) hypothesis asserts that the mean values of the dependent variable in the experimental and control groups are identical. When this occurs, we conclude that the null hypothesis has not been disproved, and thus that the experiment produced no evidence to indicate that the independent variable affected the dependent variable. This is essentially what is meant by a negative study, and it has been elevated by Cole and his colleagues who speak for the industry to the level of an affirmative finding. But a negative study suggests only that, under the precise conditions and limitations of the experiment, the dependent and independent variables were probably not related. Such a study, however, is utterly silent regarding the relationship of the variables under conditions not studied. As Edwin Carstensen (certainly no friend of those who believe powerlines create health risks) observed during his testimony in New York in 1976, “a negative study may simply mean that the investigator looked for the wrong thing in the wrong place at the wrong time.” There is only one small window of relevance for negative studies: If two studies performed under identical conditions reach opposite results, then the true behavior of nature under those conditions is uncertain. But replications are rare; none of the 11 negative studies employed by Cole to undercut the 19 positive studies, for example, could honestly be said to be a replication of a positive study. All 30 studies were different. With this narrow exception, it is clear that negative studies have no probative value—any material fact more likely than not of being true. They are simple monuments to failure (whether intentional or otherwise), and do not merit consideration as affirmative data. Texas and Louisiana contain many holes in the ground that yielded no oil. Cole's logic leads to the conclusion that there is no oil, but common sense says that there is no oil in the dry hole—40 feet away, it may be (and frequently is) a different story.

Cole has concluded that there is no truth, and he employs the art of persuasion for the ends he chooses. The good news is that neither the chicanery of a sponsor who seeks to clutter the literature with negative reports, nor the incompetence or avarice of the scientist who cooperates with this effort can actually corrode the structure of science. The careful student of bioelectricity quickly learns to separate poison-pill experiments and sophistry from facts and rational analysis, and to determine which individuals and groups are truly interested in building bioelectricity into a useful and important science, and which are interested in burying the subject under a mountain of innuendo, doubt, and disdain. The bad news is that judges and other generalist laymen, unfamiliar with the concept of the null hypothesis, may be susceptible to the Siren call of the negative study.

Andrew A. Marino