Potions, Plot, Personalities: Understand How Science Progresses and Why Scientists Sometimes Disagree
From watering hole to prime time—birth and development of an idea
Interactions between scientists, and not just disputes, play a key role in the progress of science. However, nonscientists rarely are privy to the interactions between scientists, and scientists are often stereotyped as loners. Most everyone has heard a story about a scientist coming up with some amazing insight out of the blue. Probably the most famous such story was about Archimedes leaping from his bath, and running naked through the streets shouting, “Eureka!” (I have found it!) As the story goes, he had been looking for a way to help the king determine whether his new crown was made of pure gold, or if an unscrupulous jeweler had duped him by incorporating some amount of a lesser metal. Archimedes noticed the water overflowing as he got into his bath, and it occurred to him that an object submerged in water displaces a volume of water equal to the volume of the object. He also realized that a gold crown would have a smaller volume than a crown of equal mass constructed of a less dense metal like silver, or an alloy of gold and silver. So if the crown displaced more water than would an equal mass of gold, the king had been duped. Archimedes was so excited about his discovery that he forgot his tush was bare.
Scientists rarely work in isolation
Whether the story about Archimedes’ eureka moment is true or not, it does reflect the stereotype of the brilliant scientist working alone to come up with a solution to a problem. Many scientists, and nonscientists alike, experience these sorts of “ah ha” moments while lost in their own reflections, sometimes even when they are taking a shower. Fortunately, not too many of them feel compelled to run around in their birthday suits proclaiming it to the world. However, while scientists work individually on certain tasks, they rarely do their work entirely in isolation. Neither folklore, nor textbooks, nor the media give much insight into the many levels of interactions among scientists that are so vital to the progress of science.
One form of interaction is informal brainstorming with colleagues. Like everyone else, scientists like to sit around and chew the fat. While a lot of this talk has nothing to do with science, not infrequently the conversation will get around to someone’s current research project, and the brainstorming will begin. It may explore what the results of an experiment mean, what experiment to try next, or even something as banal as where to procure a necessary device or chemical. If there is a whiteboard, blackboard, or chart paper in the room, it will soon be covered with words, graphs, pictures, and formulae. Lack of a surface to write on is no deterrent. Napkins, backs of envelopes, and paper placemats will do the trick, and if restaurant crayons are the only writing implements available, so be it. The written artifacts resulting from the discussion will simply be more colorful. Bouncing ideas off colleagues is a great way to get a fresh perspective on one’s own research because, after focusing on a problem for a while, it is sometimes hard to see the forest for the trees. Also, since individual scientists read different papers and attend different lectures and conferences, they may come across research potentially relevant to their colleagues. Furthermore, in other phases of science, scientists are expected to have sufficient evidence to back up their claims, but brainstorming with colleagues is an opportunity to get feedback on the hunches and crazy ideas that can sometimes end up revolutionizing a field. Exciting new ideas emerge when a bunch of bright people get together, listen to each other, and ask “what if?”
These informal discussions between scientists are so important that science buildings are often designed with common “watering holes,” where people go for coffee breaks or to wait for an experiment to run to completion. Different labs may share this common area, and, when feasible, buildings are planned to place research groups with complementary research interests in proximity of each other. Of course, collaboration among colleagues is not restricted to science. Many businesses design space to facilitate informal interactions among employees, recognizing that this stimulates innovation.
Answering complex scientific questions also requires more formal interactions among people with different types of expertise. For example, determining how acid rain is affecting a forest would require a biologist who knows about plant metabolism and is able to gauge the health of the trees, and a chemist who understands how chemicals in the soil (for example, metals) react under acidic conditions and is able to perform tests on soil chemistry. A geologist’s input about the types of rocks found in the area would also be valuable because different rocks (for example, limestone versus granite) are composed of different chemicals, which react differently with acid. It is therefore common for interdisciplinary teams of scientists to work together on grant proposals, projects, and papers. Even when scientists do not work together from the start of an investigation, a published study that identifies a problem—such as a new disease afflicting trees—may lead another scientist to build on the work by trying to gain insight into a possible contributing factor to that problem—such as changes in soil chemistry.
Scientists also constantly rely on tools and procedures that have been developed by other scientists. When scientists publish their results, they must carefully describe how they did the research. Published procedures are important to the progress of science because they ensure that scientists do not have to reinvent the wheel each time they want to do a new experiment. Perfecting experimental procedures is challenging and time consuming. For example, it may take months for a team of researchers to determine how to culture—grow—cells in the laboratory. Many different factors must be optimized. The cells will require special nutrients, as well as hormones and other chemicals that they would normally be exposed to in the body. Trial and error is used to determine the ideal composition of the culture medium—broth—to keep the cells healthy. Even the plates used to grow the cells must be perfected. The cells may not grow unless the plates are coated with a substance to which the cells can adhere. Finding a substance that is nontoxic and facilitates normal cell growth and division may also require trial and error. By publishing the composition of the culture medium and plate coating that promotes healthy cell growth and division, the researchers save other researchers countless hours of work, and make the scientific process much more efficient. It is not because of altruism that the researchers who do all the work to perfect a procedure make it available to everyone else. When the researchers publish a paper describing a procedure, it will be referenced in the papers of everyone who uses it. The publication of papers that are influential helps the researchers gain promotions, awards, and research funding.
Critique is very important in the publication process
While a scientist is coming up with a hypothesis to test, developing a way to test the hypothesis, and interpreting the results, close-working colleagues will provide cycles of review and feedback. Colleagues propose alternative hypotheses. They provide advice about how best to test the hypothesis, or help troubleshoot if technical difficulties arise with the experimental procedure or equipment. They suggest alternative ways of analyzing the data, such as more rigorous statistical tests. They may disagree with the conclusions drawn from the data and suggest other experiments that could be used to distinguish between alternative conclusions. If the findings hold up to scrutiny at this internal review level, then they are ready for the critical eye of outside scientists. In an academic setting—a university or other not-for-profit research center—scientists are expected to present their work at conferences and publish in peer-reviewed journals. “Publish or perish” is what young researchers are told. Scientists working in industry may also publish papers or present their results at scientific conferences, but industry scientists are often forced to keep critical aspects of their results private to protect proprietary knowledge, such as what chemicals and procedures are used to make a product or what compounds show promise toward becoming the next blockbuster drugs.
Results presented at scientific conferences are usually more preliminary than those presented in peer-reviewed journals. To give a talk at a conference, scientists, except invited speakers, must submit a summary of the findings they want to present. If the findings seem sufficiently interesting and believable to the reviewers—who are usually other scientists in the same field—the scientist will be allowed to present. Conferences give scientists the opportunity to network with colleagues at other institutions, potentially helping them set up new cross-institutional collaborations, and to get feedback that helps them prepare their work for publication in a peer-reviewed journal.
When a scientist submits a paper to a journal for publication, the journal’s editor usually sends it to three independent reviewers who make comments, ask questions, and express their concerns. The reviewers may request that the scientist do more experiments, and/or challenge the scientist’s interpretation of the results. The scientist can address the concerns of the reviewers and then resubmit the paper to the journal, unless the journal completely rejects the paper because of real or purported flaws in the science, or because the editor does not believe the paper fits with the theme of that particular scientific journal. There can be several phases of editing and review before a paper is published, and some papers will never make it to publication if the scientist cannot adequately respond to the concerns of the reviewers. The review process serves as quality control to prevent the publication of unsubstantiated claims. However, like any quality control process, it sometimes rejects outstanding work, and sometimes permits shoddy work to get through. As discussed later in the chapter, papers that are simply “before their time” may be rejected by the journal or, even if published, ignored by the scientific community. On the other hand, papers containing fraudulent data may make it past the reviewers and be published.
These flaws, while serious, need to be kept in perspective. In particular, they are not arguments against the importance of the scientific review process. A scientist’s attempt to bypass peer review by pitching a claim directly to the media is a serious warning sign of possible intellectual dishonesty. If a discovery is exciting and the data are sound, the research should merit publication in a major scientific journal. It may get published in Science, Nature, or another journal that prints articles from all fields of science, or it may get published in one of the field-specific journals, such as Cell, the Journal of the American Chemical Society, or the British Medical Journal. Either way, the published article will include a detailed description of the procedure that the researchers followed to collect the data. In contrast, when reporters from the mainstream media or popular science journals write about discoveries for the general public, they tend to skim over the details about the methods used by the researchers. Popular accounts of scientific discovery are therefore considerably more palatable than research articles in scientific journals, but they do not contain adequate information for other scientists working in the field. Without detailed information about experimental procedures, other researchers are unable to determine whether there could be an alternative explanation for the results. They also cannot replicate the results. Ultimately, it is the replication of results by other researchers that is the test of the results’ validity. Publication is not the final stage of the scientific process because when the review process fails to keep bunk from being published, future research sheds light on the error.
Arguably the most infamous example of results that were pitched directly to the media, only to turn out to be spurious, is the case of cold fusion. In the spring of 1989, Stanley Pons and Martin Fleischmann held a news conference to make the stunning announcement that they had managed to fuse atoms of deuterium at room temperature without using expensive equipment. Nuclear fusion provides the energy that powers the sun, and achieving nuclear fusion on Earth at low temperatures would be a major achievement. It would permit unlimited amounts of energy to be produced cheaply. Not surprisingly the cold fusion announcement created a hubbub within the scientific community and among the general public. The month after the announcement by Pons and Fleischmann, the American Chemical Society organized a symposium on cold fusion at its national conference. The symposium attracted 7,000 people, not a large number for a rock concert, but a huge draw for a set of talks about science. Two decades later, we do not have any cold fusion devices powering our homes or cars, nor are any on the horizon, although a small band of researchers is still working on the topic. The majority of researchers have written off cold fusion as a mistake, or outright fraud. Because Pons and Fleischmann announced their cold fusion results to the media without publishing them in a scientific journal, and they were secretive about their methods, it took time for other researchers to come to the conclusion that the signs of fusion Pons and Fleischmann claim to have seen were the result of experimental errors. Had their results been subjected to peer review before their announcement to the media, these errors would very likely have been identified before cold fusion fever spread worldwide.
In general, there is nothing wrong with scientists talking to reporters about their research. Many scientists want to teach the public about their work to inspire young people to study science and to convince taxpayers of its value. Some public funding agencies, such as the U.S. National Science Foundation, even mandate that the scientists who receive funding from them engage in activities to inform the public about their research. The problem only arises when scientists promote their research to the media in lieu of publishing it in a scientific journal, or when they make claims that go far beyond those that are supported by existing scientific research. Some scientific journals even have rules prohibiting scientists from talking to the media until right before the scientist’s paper is going to be published by the journal. These rules are referred to as the embargo policy. The purpose of the embargo is to avoid a cold fusion-like scenario by making sure a research paper is available for critique by other scientists when the popular press is reporting on the story. Therefore, claims should be interpreted with extreme caution if they have been made directly to the media, especially if other scientists are greeting them with skepticism.
The scientific review process is not flawless
The many levels of critique give the scientific process its strength, but no process is perfect. Sometimes good science does not get published, and sometimes bad science does.
Revolutionary ideas are sometimes overlooked
Barbara McClintock’s research on “jumping genes,” or transposons—bits of DNA that can move from one place on a chromosome to another—is an example of important science that initially failed to garner the attention it deserved. McClintock had collected reams of data to support her claims about transposons. She had meticulously documented how color changes in the kernels of the corn plants she bred could be linked to the changes in the chromosomes of those plants as seen through a microscope. She knew that her findings would come as a surprise to her fellow biologists, so before making them public, she spent six years collecting data to refute the objections to her findings that she anticipated other researchers would have. However, the field of genetics had not yet advanced to the point where it could provide a real mechanism for McClintock’s observations.
It took more than 20 years from the time she made her research on transposons public to its recognition by the greater scientific community. This lack of acceptance could not be attributed to the marginalization of McClintock; she was already well known for her work on the genetics of corn. Also, some other corn geneticists did recognize the importance her work, and a few even had similar findings. The problem was that in the early 1950s, when McClintock first made her work public, biologists took for granted that genes were stable. It seemed unfathomable to think that genes could jump around on a chromosome—just as scientists did not initially believe that the continents could be moving.
New data and an explanatory mechanism led other scientists to accept that transposons were real and to recognize their significance. In the decades between the initial announcement of her findings and the research community’s acknowledgement of their importance—ultimately earning her a Nobel prize—other research, including Watson and Crick’s determination of the structure of DNA, and independent confirmation in bacteria of the sort of gene rearrangements McClintock had discovered, led to a sea change in the way scientists think about genetics. They stopped viewing genes as simply beads on a string—a chromosome—and in the face of volumes of data collected by independent researchers working on different problems, the notion that genes can move around came to be accepted.
The many historical examples of the scientific community ignoring ideas that are before their time, like those of Wegener and McClintock, are often exploited by cranks to argue in favor of their implausible schemes. Their arguments run as follows:
- The scientific community is not accepting my revolutionary idea about ______ (insert topic) just as ______ (name of a famous scientist) was ignored by his/her contemporaries. Time will vindicate me, just as ______ (famous scientist) was vindicated. In the meantime, you can benefit from buying my ______ (name of product or book).
The problem with this argument is that while a number of scientists have been ignored and later vindicated, these examples are still relatively rare compared to all of the examples of individuals who put forth crazy ideas that have not been vindicated. The earth is flat. The earth is hollow. Maggots are spontaneously generated by rotting meat, and mice are spontaneously generated by linens sprinkled with a few grains of wheat. The bumps on people’s skulls provide insight into their personalities and capabilities. Christ was an astronaut who traveled back in time in a yet-to-be-developed NASA time machine. The likelihood that the ideas of self-proclaimed revolutionaries will end up on the crazy idea junk heap—along with flat Earth, hollow Earth, spontaneous generation, phrenology, and deity in a spaceship, respectively—is much greater than the likelihood that their ideas will revolutionize science. For that reason, the claim that revolutionary ideas are sometimes overlooked, while true, is a poor argument for the legitimacy of an idea.
Fraud sometimes occurs
In addition to sometimes turning a blind eye on revolutionary ideas, reviewers and the rest of the scientific community can get tricked into believing bogus results. In 2002, scandal rocked the world of physics. Starting in the late 1990s, Jan Hendrik Schön, a young physicist from Germany working at the world famous Bell Laboratories in New Jersey, and his colleagues there, published a string of papers that promised to revolutionize several fields. Just before the investigations into their work brought everything crashing down, the group was publishing at the remarkable rate of one paper every eight days, mostly in major journals. The researchers had been working on tiny electrical switches similar to the ones used in computers. They developed switches from a variety of materials and discovered that the switches had surprising properties. For example, by adding a very thin coating of the chemical aluminum oxide to the switches, they could get materials that were usually poor at conducting electricity to conduct it very well. This may not sound particularly exciting, but Schön’s papers were among the most cited papers in physics, and had scandal not erupted, his work would have very likely earned him a Nobel Prize.
But on May 10, 2002, officials at Bell Labs launched an investigation of Schön’s work after outside researchers noticed what appeared to be a duplication of data in multiple papers. Even before the discovery of duplicated data in Schön’s papers, scientists were starting to raise questions about why other labs were not able to replicate many of Schön’s amazing results, despite their efforts and the tens of millions of dollars being spent on research in the area. On September 25, 2002, a Bell Labs report concluded that Schön had committed widespread misconduct.
A few years after the scandal over Schön’s research, Woo Suk Hwang, a South Korean researcher who published pioneering work on producing patient-specific stem cell lines, was found guilty of fabricating data. Again, the problems with the work were revealed when other scientists scrutinized it and attempted to replicate it after its publication. When scientists want to pursue a particular line of work, they check their materials, equipment, and procedures by comparing their results to the published results from an identical experiment by another scientist. If time passes and other researchers cannot get the experiments to work, the original research will fall under scrutiny. Both Schön and Hwang were on the cutting edge of very hot fields. They should have known that they would eventually be found out. Had they been working on some obscure problem, it may have taken much longer for their work to have been exposed as fraudulent. On the other hand, they would not have had the excitement of making headlines on a regular basis. We will probably never know why they acted unethically, but in the end, their careers were ruined. In an unprecedented move, the institution from which Schön earned his doctorate revoked his Ph.D., although there was no evidence he had fabricated any of that research.
Although these are examples of pathological science, in the end, time and scrutiny by the scientific community did get science back on course again. McClintock’s story shows that time and the accumulation of evidence can vindicate the work of the maverick. Schön’s and Hwang’s stories show that it can also expose the charlatan. The examples of McClintock’s, Schön’s, and Hwang’s work reveal what Evelyn Fox Keller, in her biography of Barbara McClintock, A Feeling for the Organism, referred to as the “tangled web of individual and group dynamics” that defines the growth of scientific knowledge. Indeed, individuals cannot push scientific knowledge forward alone; it is through multiple levels of interactions between the individual and the group that science advances.
As Harry learned from the Half-Blood Prince’s potions book, there is a lot more to doing science than following a recipe. This chapter took that lesson further by laying bare the inner workings of the scientific process. However, Harry, Ron, and Hermione also learned that making potions was one thing, using potions on their adventures was another. Their adventures exploited Felix Felicis and Polyjuice Potion the way people who hold stake in an issue exploit scientific results for their own purposes. The production of scientific results is just the beginning of the plot. The adventure continues after the results are made public. The subsequent chapters of this book explore the twists and turns of plot that occur once scientific results make it into the public realm.