Server response survey results: Now, with more pepper spray!
Ian Lurie Nov 23 2011
Over the last three months, a bunch of folks have used my server response code tester to check whether their web sites correctly respond to broken links, correct links, etc. Per my first post about the tool, I’ve been storing the results—don’t worry, no personal data, just web site, response codes, and a grade—for a survey.
The short version: We all get a C-. I don’t know about you, but when I got a C- on a report card, it was 1 week of forced labor, doing homework inside while my friends frolicked happily in suburban paradise.
Here are the details: What I found, how I found it, and what it all means:
The response tester looks at four things:
- How your server responds when a browser visits your domain name at www.yoursite.com. It should deliver a 200 or, in some cases, a 301 response. Anything else is likely a problem.
- The tool checked the response for www.yoursite.com/sfghawerhascd.aspx. That’s a .NET page. Unless you’ve actually built a page with the address ‘sfghawerhascd’, your server should respond with a 404 code.
- It checked the response for sfghawerhascd.php. That’s a PHP page.
- And, it checked the response for sfghawerhascd.htm.
Folks tested over 2000 sites using the tool. I removed duplicates and then ran the numbers. The results are pretty interesting.
We’re all doomed
DOOMED, I tell you. Apparently, only 69% of all ‘professional’ webmasters/developers can correctly configure a web server.
The internet, as a whole, gets a D+. That means that internet marketers, as a whole, get a D+. Don’t give me that oh-this-is-technical-stuff-for-developers crap. If you can’t figure out how to correctly configure a server, you should at least be able to harass someone else until they do it.
Of 2000 servers tested, only 69% got an ‘A+’, which I defined as:
- A server that delivers a ‘200’ response code for the home page…
- A ‘404’ code for a broken link pointing at an ASPX .NET page…
- A ‘404’ code for a broken link pointing at a PHP page…
- …and a ‘404’ code for a broken link pointing at an HTML page on their site.
A D+. Next time some code nerd tries to get all high-and-mighty and cynical on you, run the response tester. If they don’t get an A+, slap them, really hard.
It is ridiculous that 30% of servers get this wrong. Asinine. Stupid. Silly. Crazy. Insane.
Here’s where the relative level of difficulty lies:
Impossible: Understanding US tax law
Really hard: Winning the Tour de France
Hard: Riding a bicycle 100 miles in a day
Medium: Riding a bicycle to work without being squashed
Easy: Riding a bicycle in a wobbly circle in your driveway
Really easy: Configuring your server to deliver the right response code
So yeah. If we can’t get this right at least 90% of the time, we’re doomed.
.NET gets an F-
.NET servers were correctly configured only 22% of the time.
Twenty. Two. Percent.
My. God. I’m a Mets fan, and 22/100 is still shocking.
548 sites tested were .NET-driven. Each site had 4 chances to get it right, for a total of 2192 chances. Total, they scored 503 points.
By the way, there’s no explanation for .NET sites’ utter awfulness. Except for the fact that .NET seems to attract awful developers like bright lights attract insects.
Mountain, not molehill
This is a mountain, not a molehill. If your server’s delivering the wrong response codes, you’re clobbering your SEO potential, and may be hurting site performance as well.
Here’s an example: When a web browser or search bot visits your home page, the best response is a 200 code. There are a few exceptions, but not many.
200 sites tested delivered a 302 response code for www.yoursite.com. I won’t explain why that’s bad. I’ve already explained it here.
When it comes to SEO, a 302 redirect on your home page is hari-kari. You’re telling search engines “Nothing to see here, move along, and don’t bother with any authority, either.” It’s a cosmic flush of the SEO toilet.
Lots of others used a 301 redirect from www.yoursite.com to /index.html or similar. That’s slightly better. Slightly.
Only 73% of servers tested delivered a 200 response.
If only 73% of drive-throughs delivered the right fast food, we’d have riots in the streets. All you have to do is ask someone to change a couple of server settings. You probably won’t even get pepper sprayed.
I’m moving to Mars
When I went into this project, I figured I’d find, I dunno, 10-20% of servers slightly messed up. Instead, I found 1/3 of the web well and truly bolluxed. I’m not sure if I should shout with joy because this means job security, or scream with despair because 30% of webmasters aren’t smart enough to breathe if they get dizzy.
On the Ian Scale, I’m moving to Mars, or opening a bicycle shop:
Mars is cold this time of year, and bicycle shops mean (shudder) working in retail again.
Please. Fix your servers.
CEO & Founder
Ian Lurie is CEO and founder of Portent. He's recorded training for Lynda.com, writes regularly for the Portent Blog and has been published on AllThingsD, Forbes.com and TechCrunch. Ian speaks at conferences around the world, including SearchLove, MozCon, SIC and ad:Tech. Follow him on Twitter at portentint. He also just published a book about strategy for services businesses: One Trick Ponies Get Shot, available on Kindle. Read More