Anecdotal Evidence is Not Evidence

  How 'it worked for me’ irks me

 

 

I hear that phrase constantly from doing what I do. I say cannabis doesn’t cure cancer; ‘it worked for me’. I say that GMO’s cannot cause an allergic reaction by replacing genomes; ‘Well, it did for me’. I say aloe vera doesn’t cure diabetes; ‘well it worked for me’. There is a phrase in the scientific skeptical community; ‘The plural of anecdote is anecdotes, not data’ – a sentence that resounds more truth that you will ever know.

How many anecdotes make for data?

How many pieces of string make a full ball of string? Well, how long are the pieces of string? What is defined as a ball of string? Think about that in context; what makes for evidence? You have a cold and you take an Echinacea tablet, your cold goes away and you put that down to the tablet, right? Is it not highly likely that your cold will go away on its own as your own anti-bodies fight off the virus, as opposed to a natural remedy? An anecdote is a story, nothing really more. As humans, we find anecdotes interesting and strive to see what works to cure the incurable. As scientists, however, we get a whiff of anecdotes and shut off. Or in my case, roll my eyes so hard I almost detach my retina. Anecdotal evidence, however much is complied, shouldn’t make for data.

The most captivating anecdotes are, of course, our own. We have a belief in something that was experienced personally, so it must be true, because we have seen it with our own eyes or felt it. It is a bias we all posses, but the skill to compartmentalise that bias is not. The reason we do not classify this as actual data is because there are so many variables. Take, for example, type 2 diabetes that is brought on by diet, which can be controlled by diet (which many people incorrectly call ‘reversing diabetes' or 'curing diabetes' – you are not, you are just controlling it. If you stopped doing what you are doing, the symptoms will return. Hence; not reversed or cured). If you are eating low carb diet and exercising after meals, your HbA1c level will decrease, meaning the complications of diabetes are unlikely to happen. Let say that you believe that this website is correct and the evidence for aloe vera was real in lowering blood sugar (it isn’t) – you then consume aloe vera alongside the diet and exercise. You could presume that the aloe is curing your diabetes, not the exercise and dieting. This makes for anecdotal evidence that shouldn’t really be entertained, but sadly is. This isn’t controlled by any means, there are so many variables its hard to begin to disseminate this particular scenario, and it is the same with most anecdotes.

Can it ever be data?

In a nutshell; no. I will hammer this point into your brain; there are way too many variables to define a treatment for something based on anecdotal evidence in things like medicine. Let’s take a less extreme example than diabetes; the common cold. You have good days and you have bad days with any illness or disease, and we have surely all experienced a cold. Where you feel okay when you get up for work and like you want to die when you get there and feel better a little later on but want to die again in the evening. The cold is what is known as a self-limiting illness, by which it will get better on its own without medicines, such as Echinacea, so feeling better is not necessarily down to anything other than the cold receding. When the cold is at it’s worst, you consume something and then when the cold levels off, you put it down to the product you consumed. As the symptoms are variable, is it the fluctuation in symptoms or the cure you have consumed?

Another factor with the cold is the availability of multiple treatments, so which treatment is had a positive effect? Was it the Lemsip or the wheatgrass? The aloe or the exercise? The cannabis or the chemo? You may have scoffed at the last one, but there are people who genuinely believe that the cannabis cured the cancer, not the chemotherapy used to destroy the cells. In fact, that last story is a prime example of anecdotal evidence that flies directly in the fact of any scientific medical evidence. A classic logical fallacy of post hoc ergo propter hoc – after this, therefore because of this. There is an excellent example of confirmation bias within anecdotal evidence, people see the information they want to see in order to confirm their pre-conceived notions. For example, people purchasing magnetic bracelets to cure ailments don’t see the absurdity because they believe it works.

Correlation implies causation is a major backing from people who rely on anecdotal evidence. As you should already know, correlation and causation have to be very carefully studied before any conclusions are drawn. Which is a major problem with media reporting on how certain things cause cancer, there’s a good chance that there isn’t actually a correlation. For an image representation of this, I found a website describing a reddit user who correlated the incline in organic food sales correspond to the diagnosis of autism. Do what that what you will, but I whole heartedly don’t think for a fraction of a second there is a link, but many people would if that word ‘organic’ was replaced with ‘GMO’.

So anecdotal evidence doesn’t hurt right?

Two words: product testimonial. You know these right? Arthur from Stockport has had problems with weight for years, he found this new natural remedy to be pushed down his throat by ridiculous claims of miracle weight loss. He’s (poorly) telling you his story of how after 4 weeks he lost 12 stone, but neglects telling you about the drastic surgery and dieting. You will have seen these ridiculous infomercials at some point. Does this hurt anyone? Well, yes, people will:

a)    Purchase the remedies, which tend to have no statistical and/or clinical significance and have an adverse reaction

b)    Rapidly loose weight by starving themselves, which can lead to further complications

c)     They don’t actually work. Which can lead to issues psychologically and cause drastic decisions, e.g. unnecessary surgery.

I could write a whole post on weight loss supplements, but I’ll leave it at that.

It would be benighted of me to not state out the obvious cases of anecdotal evidence pushed by a groups of pseudoscientific morons; the anti-vaccination movement. They use anecdotal evidence to fearmong about the development of autism in vaccinated children. They use a non-existent correlation to fight their cause and ignore the evidence; there is no link. I mean, there are countless issues with using the anecdote ‘my child got autism from a vaccine’, and I cannot even fathom to go through them. And yes, people actually die because they don’t get their children vaccinated based on anecdotal evidence. What a privileged first world we live in, eh?

Conclusion

Essentially anecdotal evidence is a story told by someone. It tends to go against the proper scientific medicine by replacing large samples for small biased samples; rely on observation rather than statistical data; discount variables as opposed to controlling them and finds trends that link to their preconceived notions as opposed to well, not. If you find yourself in a situation where someone says ‘a friend of a friend tried’ or ‘well this worked for me’ followed by something absurd, tell them to shut up or switch yourself off. Anecdotal evidence is not evidence, its mostly pushed on by people trying to sell you pseudoscience. Trust me, you don't want to buy it - see what I did there?