Analytics Apples And Oranges: Switching Web Analytics Tools Without Getting Fired
Ian Lurie Jul 8 2008
I did WebTrends a disservice here by failing to point out that my client’s creaky WebTrends installation was a very old, purely log-file-based one. Newer versions of WebTrends are every bit as versatile and accurate their competitors.
If you switch from one web analytics package to another, be ready to make some adjustments in your metrics goals.
Web Analytics Q & A
How many people here used WebTrends or something similar up until a few years ago?
How many switched to, say, Google Analytics, Omniture or Urchin 6?
OK, last question: How many of you nearly got fired when your visitors, pageviews or other metrics inexplicably plunged?
Chances are you aren’t suffering from bad statistics. You’re a victim of metrics madness: The lack of standards in site traffic measurement.
The Horror of Metric Madness
Late last year we launched a shiny new web site for a client. It’s beautiful, if I may say so.
As part of that launch, we switched them from a creaky, old, log-file-only WebTrends installation to Google Analytics (note my comment at the beginning – WebTrends is great. Just don’t use a 5+ year-old version).
Four weeks ago I got a wake-up call. Pageviews and unique visits to the site had dropped 10% over the previous year.
December 2006: 300,000 pageviews
December 2007: 270,000 pageviews
(numbers changed to protect the innocent, percentages the same)
For this client that’s pretty grim: They’d seen steady growth (thanks in part to SEO) before 2008.
Those Missing Pageviews
So, where did those pageviews go? And come to mention it, why didn’t they go up, instead of down? Have I lost my touch?
Warning: Geeky stuff ahead. If your eyes roll back in your head at words like ‘log file analysis’, skip ahead to ‘Analytics Package Conversion Factors’.
The WebTrends installation my client had used the server logs to count visits and pageviews. It counted any page load by any visitor as a pageview. Even if the visitor started to view the page and then clicked away before a full pageload.
Third-party tools like Compete.com further reduce pageviews and visits because they’re not measuring 100% of traffic to your site. They get a random sampling based on either traffic through your ISP or toolbar installations (like Alexa.com).
All of this can cause a perceived but artificial drop of as much as 50% in both pageviews and visitors. It’s often less, but it depends on the original configuration. Here are some rough percentages based on my experience:
Web Analytics Package Conversion Factors
These numbers are from sites I’ve worked on, not the entire internet. While I’ve been around the block more times than I care to count, take ’em with a grain of salt.
From Urchin using UTM.js to Google Analytics: 20% drop
From Log file-only WebTrends or Urchin to Google Analytics: 30-50% drop
From Webalyzer to Google Analytics: 20-30% drop
From Google Analytics to Compete.com: 20-60% drop
3rd-party sites like Compete.com are tougher: A busier, more popular web site may see only a small drop. A smaller, less popular one may see a very large drop. A smaller initial audience leads to a smaller sample and a larger drop.
Two Tests To Save Your Job
So, your boss or client is really pissed. It looks like you’ve cost them a big chunk of traffic. Your life, or at least your career, is about to come to a messy end. There’s plastic spread on the boss’s floor.
Here’s how you save yourself:
Go to Compete.com. Look at the trend over the time period in question.
For example, let’s say I moved my blog from Urchin to Google Analytics January 1, 2008. My own data shows a drop in visits:
These numbers are made up, by the way.
Run a Logfile Tool On Current Data
If you have access to the log files on your web site, why not point a log file analysis tool like Urchin or WebTrends at the logs? Then you can compare your visitors and pageviews across multiple analytics packages, figure out the difference and adjust accordingly.
Chances are your web hosting provider already has some form of basic log file analysis running on your server. It might be pretty crude, but you only need the most basic stats: Visits and page views.
Different analytics packages measure different ways. There is no central standard for web traffic analysis.
So, if you move from one web analytics tool to another, and your numbers plunge, do a little double-checking before you panic.
Ian Lurie is CEO and founder of Portent Inc. He's recorded training for Lynda.com, writes regularly for the Portent Blog and has been published on AllThingsD, Forbes.com and TechCrunch. Ian speaks at conferences around the world, including SearchLove, MozCon, SIC and ad:Tech. Follow him on Twitter at portentint. He also just published a book about strategy for services businesses: One Trick Ponies Get Shot, available on Kindle. Read More