SEO Analytics, Middle Earth-Style

Ian Lurie

Who saved Middle Earth?

No, not Frodo. His will shredded like wet toilet paper. Gollum took the lava bath, destroyed the ring, and saved everyone from a 36″ Dark Lord. Does he get any credit? Nooooooo. Nine-fingered Frodo is the hero. The ladies all swoon at Legolas and Aragorn. But not poor Gollum.

Internet marketers, and SEOs in particular, are a lot like Gollum: We’re shunned. We don’t get much sunlight. Our diet is awful. And we never get the credit for business success.

It’s our fault. We’re good at building rankings and building traffic. But we’re terrible at demonstrating the value of our work. So we fling ourselves into the lava. Every single time.

The way to avoid the lava swan dive? Collect big data. Present simple analysis. Don’t deliver reams of data. Use those reams of data to generate insightful, at-a-glance analysis.

I’m going to walk through whole process, from data collection to analysis and presentation, in this post:

Our sample company

Nazgul Cyclery

Nazgul Cyclery. Cycling through Mordor? We've got your rigs!

None of my clients wanted to be compared with ravening hordes of goblins and uruk, for some reason. So, I’m going to use a fake company: — Nazgul Cyclery — as an example throughout this post. Nazgul is the world’s largest bicycle shop chain. We do 10% of our business online, and the rest off. Competitors tend to, er, encounter problems, so we’ve grown fast.

We’re best known for our chainrings. There are 9 of them. Plus one forged in darkness… Never mind.

First, collect your tools

You’re going to want some tools to do all the stuff in this post. My suggestions:

  • At least one really good authority measurement toolset. SEOMOZ (affiliate link), MajesticSEO and/or ahrefs are all great.
  • A site crawler. I like Screaming Frog SEO Spider. Xenu Link Sleuth is great, too.
  • Google and/or Bing Webmaster Tools.
  • Analytics with goal tracking.
  • A rank tracking tool. Yes, we still have to look at the rankings. They go up and down. They vary. They get [not provided]. But face it: The higher ranked stuff gets more clicks. I use AuthorityLabs and Advanced Web Ranking, but SEOMOZ’s toolset can do it for you, too.
  • A basic keyword research tool. For our purposes, Adwords’ keyword tool will work OK. You can use Wordtracker, Wordze, KeywordDiscovery or something else, if you prefer.

1: Grade yourself

First, you have to answer the question: How are we doing right now?

Create a report card for your web site. Collect and track stuff that influences rankings:

  1. Number of duplicate pages on your site.
  2. Number of 404 errors.
  3. Number of 200 status pages.
  4. Number of indexed pages.
  5. Home page load time.
  6. Inlinking unique domains.
  7. Keyword diversity.
  8. Pages with no title tag.
  9. Pages with duplicate title tags.
  10. ‘Thin’ pages.
  11. Domain authority from Open Site Explorer, Majestic SEO or ahrefs.
  12. Number of top 10 rankings.
  13. Klout score, as a general social media indicator.

Stuff that doesn’t influence rankings, but you should still track:

  1. Unique visits from organic search.
  2. Non-branded unique visits from organic search. These are folks coming to your site searching on phrases that have nothing to do with your brand name or products. Someone finding my bicycle shop after a search for “fix my stupid flat tire” is a non-branded search.
  3. Time on site from organic search.
  4. Time on site from non-branded organic search.
  5. Average pageviews from organic search.
  6. Average pageviews from non-branded organic search.
  7. Bounce rate from organic search.
  8. You guessed it: Bounce rate from non-branded organic search.
  9. Conversions from organic and non-branded organic search. This one is really important. You must have some form of conversion. There’s always one — folks who contact you, a purchase, an information request. Whatever it is, track it and assign a dollar value to it.
  10. Dollar value of organic search. Also really important.

More on goals for a second: I only sell 10% of my stuff online. I’d love to do more, but my shops are important, too. So I’ll track a few things: Web sales, visits longer than 6 minutes (the site average), social media interactions and ‘find a store’ searches.

I use a spreadsheet to track it all. If you want to use my example sheet, click here. The end result doesn’t have to be pretty. This is just storage:

Site data - the initial report card

Store everything you think you'll need later

Collect everything you ever think you’ll need. Ever. You can’t go collect it later. You want a big, happy data warehouse you can use later.

2: Build the opportunity map

I’ve talked a lot about SEO opportunity gap research. Here’s a quick summary:

  1. Find phrases that generate traffic and conversions, right now.
  2. If there aren’t any, find phrases that generate traffic, right now.
  3. Check your rankings for those phrases.
  4. If you rank between the #4 and #20 positions for a phrase, record it in a spreadsheet.
  5. Map it: Record the page for which you best rank for that phrase.
  6. Record the current traffic volume.
  7. Record the traffic volume according to your keyword research tool.
  8. Figure out how many more visits you’d get if you got to position 1, 2, 3 or anything else better than current. You can use Slingshot SEO’s current research for this.
  9. Based on conversion rate, volume and potential ranking, calculate how many conversions you can gain with an upward jolt.

Now, do the same thing for top key phrases for which you have no ranking at all. When you map these phrases, you won’t know the current traffic volume or the mapped page. That’s OK — just record ’em anyway.

The result is an opportunity map, showing phrases, current rankings, current conversions/revenue and the potential lift for each. I used a little conditional formatting to highlight terms that may offer the most promise:

opportunity map

The opportunity map

Important: Yes, I’m using keywords to do this. But I’m using keywords to figure out my site’s strengths and weaknesses. I will not be telling clients “We are going to rank for blah blah.” Use keywords as a comparison tool, not a success metric. Otherwise, when keyword data goes bye-bye forever, you’ll be hosed.

3: Estimate difficulty

How hard is it going to be to make the gains you’ve mapped out? Grade that, too, using these factors:

  1. Number of competing pages with the same phrase in their title tag.
  2. Domain authority for the top 10 competitors.
  3. Social media metrics for the top 10 competitors.
  4. CPC for each phrase.
  5. Inlinking unique domains for the top 10 competitors.

I tend to do this at-a-glance, by eyeballing some of my competition data for keywords:

competition data in the opportunity map

Competition data in the opportunity map

Then I look at competition data by site, too:

Phrase data by site and SERP

Phrase data by site and SERP

If you want to get fancy, you can also track your competitors’ issues:

  1. Duplicate pages.
  2. 404 errors.
  3. Thin pages.
  4. 302 redirects.
  5. Server response code handling.

That can help you determine where you may have a competitive advantage.

This can get tedious. Learning a little Python or another scripting language will save you a lot of time. At Portent I wrote a script that grabs a lot of this for me. It saves us about 1 hour per report.

I actually assign a 1-10 difficulty score based on what I see. This is a holistic, numeric grade I assign by just looking. I find it a lot easier to organize my work once I’ve done this.

Remember: Everything we’ve collected to this point should never see the light of day. It’s your data, for your use. Don’t lob it at the client.

4: Track SEO goals based on your grades

Start tracking goals based on what you’ve found. There’s no hard and fast process for this, but I usually:

  1. Find the best possible conversion-generating opportunities from step 2.
  2. Look at my site’s weaknesses based on step 1.
  3. Look at my site’s strengths based on step 1.
  4. Look at competition for each opportunity, based on step 3.

Then I sort and prioritize. Some ultra-competitive phrases may offer huge traffic and conversion gains. Those become long-term goals. Other less-competitive phrases may offer solid traffic and conversion gains. Those become my near-term goals.

5: Map actions to opportunities, based on your grades

What’s it going to take to move up for these goals and grades? You now have a lot of data. You can use it to make some pretty solid decisions. Look at each opportunity in your map. What do you need to do to pass your competitors?

You need to map specific actions to these opportunities, so that your team knows what to do, and the potential impact.

  • Great link profile + thousands of duplicate pages = You’d better fix those duplicates. Provide specific recommendations for fixing each duplication problem. Track completed recommendations.
  • Thin content + competitors with great content = Content strategy. Provide an editorial calendar and publication guidelines. Track launched content and response.
  • Lousy authority + competitors with high authority = Social media and link acquisition. Provide curation guidelines and start a link campaign. Track links acquired and social media profile growth.

Again, look at how I’m using the keyword data: Not to measure success. Instead, I’m using it to drive decisions about next steps.

This is the first bit of information you’ll show the client. This is the ‘what’—what you’re going to have to do to grow.

6: Track it all

Every day, update your tracking sheet. If you’re short on time, do it every week. Or, find a way to automate it all.

My favorite tools for automation are Google App Script and Python. Use what you like.

Whatever you use, make sure you update your site and goal data on a regular basis. A real masochist will also update the data on all competitor sites. But I train using a video series called TheSufferfest and I still don’t do that, so it’s totally your call. If you have underlings to do your bidding, go for it.

7: Create the report card

All of the work you’ve done so far sets the stage for the report you’ll show your bosses. Right now, you probably have spreadsheets with a lot of columns and even more rows. You can’t send that to the Nazgul Cyclery VP of Marketing. I hear he’s a real hardass:

My clients are all cats, so I’m in really good shape. I won’t bury them in all the data because I like them.

You want the report card to link action to results. That’s the entire purpose of this exercise. So my dashboard would include:

  • Sales.
  • Change in SEO issues, such as duplicate pages. Improvement should mean good things, after all.

Here’s my result:

The Dashboard

The dashboard - all that work, for this?!

That’s it. Keep it simple. If you need a more complex dash, go to step 8.

A ‘report card’ is just a style of dashboard. There are lots of great resources on creating good dashboards. Take a look at the examples on for starters.

8: Layer your dashboards

You may have multiple layers of bosses.

If so, create additional, more complex dashboards as needed. Track changes in individual social metrics, keyword diversity and other stuff. Allow your audience to drill down through the data.

9: Don’t report. Analyze

Don’t just hand over the report every week/month!!! Little misunderstandings can lead to big consequences.

Include good, useful analysis.

Good analysis:

  • “We fixed all duplicate content issues and added 10 inlinking domains last month. We now have 300 more indexed pages, and our domain authority rose 2 points. The result? We’ve got 10% more non-branded traffic and sales from search are up 2%. Next action: Step up the content campaign and start our next Smeagol Sweepstakes.”
  • “Our site had a serious duplication issue last month. That meant more indexed pages, but poorer rankings. As a result, sales fell 1%. Next action: Fix the duplication issues. Potentially dump guilty parties into Mt. Doom.”
  • “The last blog post we did was featured on Overlord Weekly! We gained 25 inlinking domains. That hasn’t kicked up the rankings yet, but it will. Next action: Figure out why that particular post attracted so much attention and adjust our strategy. Also, let folks know by publicly thanking Overlord.”

Bad analysis:

  • “Rankings are up. Sales are up.”
  • “Traffic fell 10%, but we expect it to recover.”
  • We’re all doomed!!!! Dooooommmmmmmeeeeed I tell you!!!

Last advice

Don’t be afraid to brag! Most SEO’s (including me) are so paranoid about over-promising we never take credit for anything. Get over it. Take credit when it’s due. One great way to do that is to give credit, like this:

“Thanks to the dev team, who put in a lot of extra time with me to figure out the duplication issues, we fixed all duplicate content issues and added 10 inlinking domains last month. We now have 300 more indexed pages, and our domain authority rose 2 points. The result? We’ve got 10% more non-branded traffic and sales from search are up 2%. Next action: Step up the content campaign and start our next Smeagol Sweepstakes.”

Also, put everything in context. Say the company is investing $50,000 a month in content strategy (I stifled a giggle right there). Your team produces 25 great pieces each month. That’s, uh, $2,000 per item if you do the simplest math (and everyone will). Yikes.

But that one piece that got into Overlord Weekly generated 25 great links, at least 1,000 new followers, and $5,000 in sales. Compared to the banner ad campaign, or the direct mail campaign, that’s a fantastic result. Be sure your boss or client knows.

Most important: Stick with it. A dashboard with gaps and inconsistent data is as believable as Nice Smeagol. Keep it in mind.

Start call to action

See how Portent can help you own your piece of the web.

End call to action


  1. Great post, Ian. You’ve got a very thorough process here. Bookmarking this post for more in-depth consumption in the near future. đŸ™‚

  2. I like the idea of mapping domain authority against sales.
    How would you best account for slow updates to DA scores or shifting of scales used to calculate the score?
    I’ve found clients like their reports in the same, hopefully useful to them, format and on a regular schedule. Relying on third party data can result in interruptions to this schedule or verbose explanations as to why the numbers have changed…

  3. Very nicely laid out, Ian! I can definitely see the value in having automated the process as much as possible. Sadly, I’m still learning my php… python is definitely down the road a bit for me.

  4. @Ryan that’s often a problem. We generally use 2-3 different tools so that we can provide averages, switch if necessary, etc. I also typically hide that data unless the client REALLY needs it, or we have a HUGE problem with links.

  5. Number 9 is the best advice of all – don’t just report, analyze. It’s too tempting sometimes to just quote numbers at clients, and expect them to come to same conclusions you would. Give them the conclusions, but have the numbers to back it up.

  6. Hi Ian,
    Great article, I’m going to create a dashboard including all that points. Only one doubt. How do you track duplicated pages? Do you use any specific software?

Comments are closed.

Close search overlay