Science’s Reproducibility Problem. The Cost of Science Errors. Science, Now Under Scrutiny Itself. Recent headlines communicate a crisis in credibility for researchers. And with stories like the 2009 retraction of a dozen crystal structures from a single author, biophysics is no exception.

Implications of this broad problem for our field were the subject of Sunday’s afternoon panel, Transparency, Reproducibility and the Progress of Science, cosponsored by the Biophysical Society Public Affairs and Publications Committees. Part of the Professional Development and Networking series at this year’s Biophysical Society Annual Meeting, the 90-minute program was moderated by Publications Committee Chair Olaf Andersen (Weill Cornell Medical College), and featured three speakers with similarly substantial authority in both research and administration.

Keith Yamamoto from the University of California, San Francisco opened the program with a breakdown of four major challenges to reproducibility in bioscience. Although difficult to quantify, he estimated the contribution of willful misconduct to be relatively minor; more substantial in his view are experimental errors, statistical insignificance, and the inherent complexity of biology. As Vice Dean for Research and Vice Chancellor for Science Policy and Strategy at his institution, it is perhaps unsurprising Yamamoto collapses the majority of these problems into an education deficit: a gap in scientific training, particularly at the graduate level.

Yamamoto envisions doctoral curriculum reform that takes responsibility for broad scientific literacy, with a rigorous regard for quantification, statistics, exposition of known variables—and acknowledgement of the Rumsfeld factor, the plethora of unknown unknowns in our discipline. As a goalpost, he quoted a seminal paper on the genetic code—now over fifty years old—in which Nobel laureate Marshall Nirenberg admitted with brutal honesty his own challenges reproducing critical findings. Along with tackling correctable sources of error, Yamamoto called on scientists to restore a more open humility about the complexity and variability of biology itself. Easy as it is to identify sources of external pressure and regulation from publishing and funding agencies, he emphasized the responsibility lies with scientists to change our own culture.

Emilie Marcus, Editor-in-Chief of Cell and CEO of Cell Press, echoed much of Yamamoto’s message from the perspective of the publishing industry. She implicated an even longer catalog of causes in irreproducibility, and identified five major ways publishers can contribute to restoring credibility. The first of these, methodological transparency, has motivated the recent creation of repositorites like Nature Protocol Exchange and Elsevier MethodsX; other priorities for Marcus include data sharing, ethical evaluation, accountability in the review and retraction process, and training efforts—like this panel. In 2014, Cell endorsed the NIH Principles and Guidelines for Reporting Preclinical Research, with the mission of “identifying the common opportunities in the scientific publishing arena to enhance rigor and further support research that is reproducible, robust, and transparent."

Responding to audience concerns about the limited incentive to report negative results, Marcus pointed out that Cell Press regularly publishes carefully executed findings that challenge previous data; a greater concern, she said, is that many “failed experiments” tell us less about biology than about the inherent challenges of research. This issue is likely exacerbated for the life sciences: in contrast to some mathematical fields, a biology reviewer can rarely reproduce another author’s experiments in their entirety. While acknowledging the roles publishers can play, Marcus insisted pressure for funding, publication, or tenure cannot become an excuse for unethical conduct among practicing scientists; again, culture change should primarily come from within.

Helen Berman brought many of these points home to biophysics, speaking both as a Distinguished Professor at Rutgers University, and as former director of the RCSB Protein Data Bank (PDB). The sensational Murthy retractions made not-so-welcome headlines for the PDB in 2009, and partly motivated the recent adoption of the more rigorous PDBx standards for macromolecular structure data deposition. More broadly, Berman cited the PDB as a touchstone for bottom-up collective action within the science community. She drew parallels to the ideas of Elinor Ostrom, a 2009 Nobel laureate in Economics, in emphasizing the sustainability of this approach over top-down enforcement; but she credited crystallographer JD Bernal with an even earlier insistence—articulated, among other places, in his 1939 text The Social Function of Science—that scientists work together to share practices and resources.

Following the panelists’ remarks, discussion opened to a range of questions—some emotionally charged—around what has changed in the credibility of science, and what needs to. Some contention surrounded the Biophysical Society’s own decision not to endorse the 2014 NIH Guidelines; the Biophysical Journal did develop its own Guidelines for the Reproducibility of Biophysics Research, reflecting similar principles as the NIH list, but dodging the murky “preclinical” label. A more general refrain was the evident deficiency in training of new scientists—though its causes and potential remedies were less clear. As Marcus pointed out, every manuscript published in Cell includes at least one senior, experienced author; the artisan-model of science training should provide infrastructure for younger authors to learn from these mentors. Still, several participants commented on the changing character of academic labs: as Yamamoto put it, "we’re not really training students—we have a bunch of people doing our experiments.” Others argued the culture of scientific discourse itself has changed: conferences (including this one) rarely feature true work-in-progress posters these days, nor forums to workshop experimental design.

It seemed appropriate, somehow, to tackle these topics in Los Angeles on the day of the Oscars, in the midst of a wider national debate over credibility and authority in the very different Hollywood entertainment industry. If definitive answers were not in evidence, the transparency of conversations like these still seems critical to navigating a brighter path through crises to come.