... So what were my observations .....
...... to be continued
- Reproducibility. This is desirable all across science and there are a number of reasons that make it important. Experiments stand or fall based on whether they can be reproduced. However, often, the precise conditions of measurements are not clearly stated (and often absent) in scientific papers. This, as well as the inappropriate control measurements. Some scientists complained that (like the current status in popular media) 'new science' is sexy and journals will often publish something very exciting at the expense of quality. Essentially, time is the arbiter of whether a piece of science gains acceptance. One person suggested that a repeat experiment (of another group) should be published in an open source format (very simple to do), and builds on the credibility of the experiment or theory.
- Nothing is measured in isolation! Whether you’re measuring the conductance of a single molecule or the vibrational modes of a crystal or the reactive nature of an enzyme you are using tools and instruments. Those tools are limited and in some cases, in certain fields, measuring the ‘same thing’ in a different set up can yield different results. A scientist’s job is to be transparent about methods. Some groups, however, when publishing breakthrough work, withhold methods to block others from entering the field; it gives them a grace period before others can be involved in that same niche of experiments. It’s important to control for your experimental method, by measuring in different ways.
- Practically speaking, scientific groups, however, cannot publish a repeat experiment of another group (no journal would do that). This is where conferences are important and scientists DO tell other scientists "we cannot reproduce your results". The result of this is either: 1) more details are needed in that experiment, or 2) the experiment was flawed. Both are possible...
- Issue: every scientist will select interesting data to publish. But don’t think that scientists are being misleading by doing this; most/many (?!) are responsible and publish what their experimental yield is, so you can judge for yourself. "Yield" is essentially what percentage of the time does your 'stuff' actually work, but must be carefully defined (!) e.g. “my experiment showed cool stuff 10% of the time” is different to “of the 100 experiments, 10 showed something but only 1 showed cool stuff” –both could be presented as “10%” yield: be careful. ‘Yield’ is also analogous in our everyday lives: How much of your own work day was productive that you would report it in a minute-by-minute report of what you did?
- In science, often the things that DON'T work, are very very important. In my own work, I'm quite certain that I've repeated failed experiments of scientists of the past, which was perhaps avoidable had I known that it’s been tried before. In fact, I remember finding an old lab book from 15 years ago (in our lab) and finding details of a failed experiment that I was just about to do myself! This is why its important to speak with others who are in the field to see what they’re doing, what they’ve tried and paint a vision for the future.
...... to be continued
RSS Feed
