• Question: How do you know if the tests you are doing are valid? because there is always someone who disagrees with you! =(

    Asked by lucasjacobs to Meeks, Pete, Stephen, Steve, Tom on 16 Jun 2010 in Categories: .
    • Photo: Marieke Navin

      Marieke Navin answered on 15 Jun 2010:

      That’s how science works – you publish some results and people may disagree with you but the experiments will tell you the answer! it’s the competition between scientists that drive science.

    • Photo: Stephen Curry

      Stephen Curry answered on 15 Jun 2010:

      Cor Lucas – definitely question of the day in my book! (And were you the one who was asking about magnetism?).

      This cuts to the very essence of science – how do we know if our answers are right? First off, it is important to be highly imaginative, so that you can try to think through your experimental design and spot flaws before you begin. It is important to make sure that you include plenty of controls so that you know if any signal that you observe is significant (or above background).

      But even then, once the results are in, how can you be sure?

      Well you can’t. But science is a community activity so your colleagues and competitors are there to ‘help out’. You write up your results and send them to a journal for publication. The journal will send your paper out to other scientists who are experts in the same area and will look at it very carefully. They might say it’s fine, or ask you to do some more experiments (or change some words) or — worst case scenario — they will say it’s rubbish. In the last case, the journal will refuse to publish the paper – you have to go away and try to do better.

      But even if your paper is published, is it right? As the years go by and people read it, your results are likely to influence the work of others (who may be studying a similar problem – for example, how nerve signals are transmitted between cells). If your work is right, then future experiments based on it are likely to work too, confirming your discoveries. But if your work leads someone else down a blind alley, that may suggest that something is not quite right with what you did. Maybe you made a mistake. Maybe there was something that you simply didn’t spot or couldn’t detect (because your equipment wasn’t sensitive enough)?

      Naturally enough in some areas there are strong arguments between different research groups. Scientists can be very combative at times! The theory of prion diseases —diseases transmitted just by proteins (not viruses or bacteria) — was contentious for a long time (and still causes some arguments).

      But the great thing about science is that even if someone is wrong, the scientific community will eventually come up with the right (or at least a better) answer.

      And that, long as it is, is only a partial answer!

    • Photo: Tom Hartley

      Tom Hartley answered on 16 Jun 2010:

      Hi Lucas,

      Excellent question, it gets to the heart of what science is about. If someone disagrees with me (or if I disagree with someone else) it’s up to that person to come up with some evidence to change my mind. If there isn’t any evidence, then one of us should run an experiment that will settle the matter one way or another. If they can’t come up with any evidence or an experiment, there’s no reason for me to change my mind.

      This approach works really well for some questions and not so well for others. For example it’s good for questions like “Does this medicine really work?” and it is not good for questions like “Is there life after death?” (because so far as I can tell, there is no way of getting evidence one way or the other). With that kind of question there’s no way to know whether you’re right or wrong, but scientist generally prefer simpler answers which don’t rely on imagining complicated things we can’t see or measure in some way.