Owls are not what they seem

These days I have the opportunity to watch live the TED conferences. Today, computational neuroscientist Sebastian Seung, while talking about connectome said something like “…when we’ll be able to test and prove…” At that point I had a flash: two words that sounded wired together. We can test but can we really prove?

As a statistics person it just crossed my mind that every experiment designed to prove a theory has two flaws:
1) it is done under controlled conditions;
2) even endlessly repeated, the experiment’s results are gathering into what statistics calls a sample.

So what does it means? Let’s say a new stomach drug is tested. The scientist is reproducing the environment existing in the stomach and then drops the drug to see its action. Maybe the drug acts as expected but are we really sure that those precise conditions are repeating identically in the real world? Probably not as we are different, even if the differences are tiny. Moreover, some of the experiment’s conditions might escape the scientist, like the reaction was in the presence of light (absent in a real stomach!) or the missing saliva which can alter the substance or the reaction time.

Even repeating the experiment several times with real people you cannot say you are 100% sure the drug works or not. Because no matter how many people would use that drug, they still remain a sample of an infinite population (of people and of the same people in different moments in life), the result will be just a variable and not the parameter. This means that we can tell only that “the experiment has a probability of x% to be successful” But x will never be 100. And the difference is the margin of error.

In any science it’s the same. Everything that was proved through experiments it’s just a statistics. And yes, the margin or error may be so small that we don’t even care to acknowledge it but still exists.

<<.Previous | Back to summary | Next.>>