Ian Lurie // Dec 31 2009
You may think that, with Google Analytics, you don’t need your server log files any more. Or, as someone who runs a web site for your business, you may not even know what log files are. They’re redundant, right?
You still need those log files!
Whoa there, you say. Ian, you’re going geeky. What the heck is a log file?
It’s your server’s record of who’s come to your site, when, and exactly what they looked at. It’s incredibly detailed, showing:
If you were to look in a raw log file, you’d see something like this:
I know: Blech. But stay with me.
Say you want to find every page on your site that received zero visitors from organic search in the past 2 years. That’s a useful statistic: Those pages may be invisible to search engines, or they may be really poorly optimized. A list of organic search outcasts could keep you busy for quite a while.
You could use Google Analytics to find those pages – I’ll be writing about that next week. But only if you’ve been using Google Analytics for 2 years, and only if you had it configured correctly, and only if you can rely on Google Analytics’ organic click detection (which isn’t always 100%).
Or, you can find a geek like me, have them write a script, and zip through all of your files in a few minutes. Then you get a nice, neat list like this:
That’s every page and image that hasn’t received a click from an organic search result.
If you have this data, the cool scenarios are endless:
And so on.
If someone, say, deletes your Google Analytics account, your log files are your fallback.
If you switch from one analytics platform to another, you can see a big rise or fall in traffic or pageviews. Different platforms use different measurement methods, and that makes for inconsistencies.
But that’s really hard to explain to your boss when she asks you why the new site lost 30% of the old site’s traffic.
Analyze the log files, instead, and you can get a consistent view of traffic that spans both the old and the new analytics packages. Problem solved.
…make sure you’re keeping those log files around. And make sure they’re recording every available variable, including:
Most important, don’t let your webmaster delete old log files. Lots of hosting companies do this to ‘save space’. But it’s pretty easy to compress old log files to a far smaller size. Even if your log files are huge, you can store them on a humungous hard drive or use a service like Amazon S3.
Don’t. Delete. Your. Log. Files.
If this is all total gibberish to you, don’t worry about it. Just make sure you keep your log files around, and that they’re complete. Wave at your IT guy and say ‘make it so’.
You’ll thank me later.
Ian Lurie is founder and CEO of Portent Inc., an internet marketing agency that has provided internet marketing, including PPC, SEO, social and analytics services, since 1995. Read More