“Not Provided” is old news by now, and most SEOs have learned lots of different keyword discovery strategies. There are costly solutions like mining paid traffic and using third party tools, or just plain complicated tactics like correlating ranking data with organic landing page traffic. These are all great and people absolutely should continue using them, but it seems like no one is talking about the most accessible solution out there: Google’s Webmaster Tools.
While the data in it hasn’t always been the greatest (Ian gave his opinions about Webmaster Tools’ reliability a few months back), Google rolled out a fairly quiet update at the beginning of 2014, and started giving us “exact” impressions and clicks (more on those scare quotes in a minute). So let’s go over what Google is giving us with this free tool.
Why is Webmaster Tools so great?
For a couple reasons. First, WMT is dead easy to install. In fact, if you have Google Analytics installed, you just need to set up a profile. If you don’t, there other options like uploading a single file or adding a single line of code.
Second: it’s free. I use SEMrush all the time and the data it gives is invaluable. But I am also lucky enough to work for an agency that pays for my subscription. WMT gives you great data that you wouldn’t be able to get for free anywhere else.
Lastly, it tells you how Google is crawling and indexing your site. If there’s a problem or you get penalized, this is how they’ll tell you about it. It also gives you information about how your site is being displayed in search results, lets you submit a sitemap, and much more.
For now, though, let’s just focus on the keyword part of WMT.
Where is this Keyword Data?
The section you’re looking for is called Search Queries, which you can get to from the Search Traffic section of side navigation or the main dashboard. In there, you’ll see two tabs.
The first tab is Top Queries, which is basically the raw list of keywords that drove traffic to your site, sorted by Impressions, Clicks, CTR (click thru rate), and even Average Position (which, surprisingly, is usually accurate). Using the filter option, you can filter by search type (Image, Video, Mobile, Web, or All) along with region and amount of traffic. The default for the search filter is “Web” but, for the most comprehensive data, I recommend switching that to “All.” Unfortunately, you’ll need to do this every time you navigate away from this screen.
The next section is Top Pages, which will give you the URLs and data about the impressions, clicks, CTR and position for each page. But the best part here is that clicking on a URL will give a drop-down of all the terms that the URL showed up for!
Now there’s no data passed between this and Google Analytics, so you can’t actually track behavior through this, but at least you have access to some raw data and potential keywords that you may not have been aware of.
How accurate is this data?
The best answer I would give to this question is either “meh” or some word that uses the suffix “ish.”
Ian’s post was written before the update, so it’s a little better than it used to be, but using the word “exact” is a pretty big stretch. Now that there’s been a bit of time since this went live, I decided to do an actual comparison between “exact” WMT data and Google Analytics visits.
To do this, I pulled data for a few sites and compared non-paid search visits from Google Analytics to the number of Clicks from Webmaster Tools for the same period. The results were actually pretty well summed up by this snapshot of Portent’s own organic traffic and click data.
At first glance you can see that they kind of sort of match up. But then you see things like that -64% difference. Yowza!
But really, it’s not that awful. First off, we know that both are pulling from a sample, so some discrepancy is to be expected. But from a trending perspective, you at least see that the amount of traffic each page gets roughly breaks down in the same manner.
What not to do with this data
The most important thing to remember is that this isn’t the same as the old organic keyword data. You don’t want to give this to your client in a nice spreadsheet with one column named branded, the other non-branded, and tell them that they got 424 organic clicks in February for “Nerf bats.” The data will never match up and you’ll be spending most of your time explaining where these numbers came from and speculating on why a click may or may not have triggered a visit in Analytics.
So what can we do?
If it’s not exact, then what is it for? Well, this is where you can do some keyword discovery. Remember how you used to find some traffic to your site from a term you never thought of? Then you would yell “SEO powers activate!” and start focusing on that term, seeing those visits rise!
Well, now that’s back, to a certain extent. Let’s look at the organic keywords from Analytics for a specific page on Portent.
98% not provided! Are you kidding me? And out of the 638 visits, the most popular keyword has one visit. Now let’s look at the same page in Webmaster Tools.
Hot dog! Look at all those keywords. Before the update, WMT would just give you a list of terms with “< 10” clicks, and impressions rounded to the nearest thousand, which wasn’t terribly useful. Now we have exact(-ish) counts to sort by. Unfortunately, that neat little drop-down won’t export through WMT, but you can just copy/paste it right from your browser and throw it into Excel. Then you can copy that same list of keywords into Keyword Planner (Google’s other amazing free tool) and get search volume. If you want, you can even pair this with ranking data from your favorite rank tracking service or Webmaster Tools itself.
Now you have new terms and old school style.
This isn’t perfect. In fact, just looking at the drop-down, you can see that the number of clicks it gives for each keyword typically won’t add up to all the clicks for the page. But it’s something. And something is better than nothing.
What do you think of the new Webmaster Tools data? Have you found any amazing research tricks of your own with WMT data? If so, comment below!
This is exactly the method I’ve been using for keyword discovery! Hat tip to Brian Dean, if you export all your queries you can look at your queries at the bottom of page one or the top of page 2, get the search volume, and choose the best ones you’ve got some good keywords ripe for a tiny bump in optimization.