Of flying pigs, the perfect experiment, and fatally flawed dogma. A response

Caveman


The following was left at the entrance of my cave recently. It is a response to a recent Sticky Wicket, "I wanna be like Mike (or Gary or Fiona)". Because so many important points were raised, I felt it worthwhile to discuss them openly at some length, because this is the sort of discussion we should all be having.

Dear Caveman

Re: I wanna be like Mike (or Gary or Fiona)

I disagree. Science is done by people. Pigs do not throw themselves out of windows, and it’s important to know that the experiment is flawed if Prof. Snotely or whoever threw them out of the window. Snotely may throw farther than his assistant, for example, making the experiment ultimately unreproducible. It also would not occur to the pigs to measure the duration of their flight as they are plummeting down the side of a building.

I think presenting data with the perspective of remembering that people are imperfect and that therefore there is no such thing as a perfect experiment is absolutely necessary and very educational! Besides, we need to be prepared for the political mess that awaits us when we eventually do leave graduate school.

Science is by no means objective, and presenting the data as if it is ‘absolute truth’ merely ‘liberated’ by our able hands leads to long-term misinterpretations, to the extent that some ‘classic’ experiments become dogma even when they are fatally flawed. Our job as the young up-and-comers is to look for those places where the dogma has holes in it or seems inconsistent with more recent experiments. That’s what we’re going to build our careers on, or we’d never be in biology at all - we would assume that everything had already been done!

Samantha Zeitlin


 
Dear Caveperson Zeitlin

I am pleased that you took umbrage at the Sticky Wicket "I wanna be like Mike (or Gary or Fiona)", because this is the best way to discuss the issues, and you certainly have quite a few!

I do not disagree with you that science is "done by people" and that "people are imperfect". However, to me, that is why it is important to have experiments, data and results written up in a complete, scholarly manner and published in journals for assessment by the rest of the scientific community. Of course pigs do not know whether they can or cannot fly, and cannot document how far they flew, at least no more than an antibody can document that it has bound to the right antigen. Therefore, it is the job of the experimentalist to design the experiment appropriately and report the results in a factual manner. If Phlemspengler and Snotely reported that part of the experiment was that pigs were tossed from the window in order to initiate flight, then we as critical scientists could evaluate the experiment and results in that light. As I stated in the Sticky Wicket, "However, it remains to be shown whether these pigs can sustain powered flight or simply glide after an assisted launch" - those comments are related directly to the facts as presented.

Would you not agree that reproducibility of experiments and results is the cornerstone of scientific progress? I have argued from the very beginning (see "Who am I? It shouldn’t really matter.") that it is not the individual doing the experiment that is important but rather the approach and results that are. We should access results coldly as if we had (or could have) obtained them ourselves, rather than as if a famous scientist obtained them (and feel therefore that they must be right!). If you consider that the latter is not the case, in other words take a less objective view of the experiments and take into account who did them, then in my opinion the scientific process has broken down. Perhaps this is what you mean by "Besides, we need to be prepared for the political mess that awaits us when we eventually do leave graduate school." I too worry about this, as it seems to me that the politicization of science can make it very difficult to combat entrenched dogma - but then again, that is why it is important to speak out in opposition.

"There is no such thing as a perfect experiment." I think that some very reductionist biochemical experiments are perfect (e.g. showing the action of an enzyme on a substrate and the formation of a product), so are some genetic experiments (e.g. the deletion of a gene leading to a specific phenotype) and structure analyses (e.g. protein structures at a couple of angstroms resolution). Are the vast majority of cell biological experiments perfect? No. That’s the problem (and fun) of working with a ‘dirty old cell’. That is why the interpretation is important. I agree that the interpretation, also called a personal, prejudiced view (also called the ‘Discussion’ of a paper), should be viewed with caution. So, when recalling the work of others, it is important to state the facts (i.e. the experiment and data) rather than the conclusion of the author of the experiments and results.

You state that "science is not objective". I respectfully disagree. The recitation of an experiment and the subsequent data should be objective and the "absolute truth", in my opinion, as long as it is done carefully and documented properly; repetition of the experiment by someone else (recall, a cornerstone of the scientific process) will determine the veracity of the report. Yes, I agree that experiments can lead to "misinterpretations... and some ‘classic’ experiments become dogma even when they are fatally flawed." However, I believe that the cause of this is a lack of open criticism by other scientists, the prevalence of mini-reviews, short summaries of papers, etc., which allows us to be lazy and not read the papers in detail, and a societal trend not to dispute results and interpretations publicly.

"Our job as the young up-and-comers is to look for those places where the dogma has holes in it, or seems inconsistent with more recent experiments. That’s what we’re going to build our careers on, or we’d never be in biology at all... we would assume that everything had already been done!" I have to admit to having felt despair as I read your rock-mail until I got to these last couple of sentences. Now I have hope. You want to know about the work of others that have come before you, to carefully critique that work and the latest experiments to draw an interpretation (that is yours), to examine the experiments regardless of who had performed them, and to appreciate that our knowledge is very incomplete. That’s what science is all about. You get it! Just make sure that you pass it on!

    Caveman


 

Dear Caveman

Of course I disagree that reductionist biochemistry or structural experiments are perfect. If I thought so, I’d have stayed in biochemistry or crystallography. Instead I choose cell biology, because I’d rather do imperfect experiments on a ‘real’ system than clean, aesthetically pleasing experiments that have very little basis in biological reality. We should be careful not to confuse ‘clean and neat’ with ‘perfect’. In vitro biochemistry, to some extent, and crystallography are really just model building, and only a step above computational theoretical work when it comes to reality-based experimenting (much as reality-based programming is putting people together in an artificial environment and doesn’t much resemble ‘real life’). Granted, I think in vitro biochemistry and structural studies are crucial parts of science, and I’m glad someone else is doing them, so that I can take advantage of kinetic studies on inhibitors I choose to use, for example.

The real problem is that we don’t all have time to read every background paper on every reagent that has preceded what we now have before us. The bulk of literature has become too great. We do the best we can, but since we’re people, and not Jedi Knights (I’m remembering Yoda saying "There is no try, only do"), we’re never going to be able to read everything unless we stop doing experiments! There simply isn’t time to strive for perfection at everything. You have to pick your battles. I read as much as I can, but I’d rather be at the bench!

When I say science is done by people, I want to say I prefer the old papers - Meselsohn and Stahl, the famous Watson and Crick papers - and I prefer them because they write in the first person. All observations are inherently biased when made by people or by computer programs that were written by people. Call me cynical, Caveman, but it’s true. I like being reminded that someone was holding the pen.

We can’t depend on independent confirmation. So few experiments these days are ever repeated by independent groups, since the real glamour is in publishing something first, in being the pioneer, etc. In cell biology, everyone hordes their antibodies, for example, because they don’t want to risking losing their place at the front of the line. At best, things are done over again in another organism, bringing into contention the idea that ‘everything’ will be ‘conserved’ from organism to organism - of course Darwin hasn’t been born yet, and so maybe we shouldn’t predict that particular dogma just yet. Gives us all plenty of room for rationalizing our discrepancies, doesn’t it (maybe worms and flies are different and that’s why it’s a different kinase, etc.)? Anyway, you’ve probably written on the topic of ‘publishing first’ and how that has drastically influenced our objectivity, but I confess I’ve been too busy reading minireviews to keep up with all your articles!

Caveman Sam (Samantha Zeitlin)





This Article
Full Text (PDF)
Alert me when this article is cited
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in PubMed
Alert me to new issues of the journal
Download to citation manager
Google Scholar
Articles by Caveman,
Articles citing this Article
PubMed
PubMed Citation
Articles by Caveman,