5 Kinds of Bad Statistics, and How They Can Kill Your Internet Marketing Campaign

traincrash Analytics

Ian Lurie May 6 2008

train crashes through station - yeah, but our alexa numbers are great
Beware bad statistics. They can kill your internet marketing campaign with bad assumptions, incorrect assessments, and by sending you far off track.
By ‘bad’, I mean:

  1. Statistics that are collected from a small population. Alexa, for example, collects a lot of their data based on Alexa Toolbar users. It’s very possible those users behave differently than the rest of the internet population. So being top 10,000 on Alexa may not mean a heck of a lot.
  2. Statistics that are just a rough model of the real data. Google’s toolbar pagerank (the pagerank number you see in the Google Toolbar) is based on the true pagerank. And, by Google’s own admission, it’s totally inaccurate. Don’t obsess over it.
  3. Insufficient samples. “9 out of 10 visitors to our site made a purchase. We had 10 visitors”. ‘Nuff said.
  4. Subjective data. “90% of our visitors really like our site.” Really? How do you know? Did you watch them through their monitors?
  5. Statistics collected in focus groups. Focus groups are worthless, in my opinion, for gaging visitor response to a site design or an interface.

Avoid bad statistics.
Even no data is better than bad data – then you know you’re flying by the seat of your pants. With bad statistics, you’re flying with no pants.

tags : conversation marketing

5 Comments

  1. This is a great list. I’ve found more than half of my new clients will often be relying entirely on at least 2 of those you listed, and then wonder why their online marketing strategies are not bringing in fruit.
    Maria Reyes-McDavis

  2. It’s funny how statistical slight of hand works on individuals that are paid to be much smarter than I am. I sit in meetings and watch the term “average” thrown out without any qualification and no one questions it.
    Classic book on the topic: “How to Lie with Statistics”.

  3. Ian

    50% of clients love good statistics. The other 60% aren’t so sure. Or something.

  4. I thought Alexa page ranking had altered recently according to a number of sites that I have seen, where it is no longer based on just Alexa tool bar info and is now on a wider plane… not that this would make it the best in the world at giving accurate statistics but would this not mark an improvement?

  5. Ian

    @sdenterprise I think the jury’s still out on that. We can hope so, though….

Comments are closed.