It keeps getting worse! The October 2011 Google SSL search privacy update effected the lives of SEO professionals everywhere. It made things a lot more challenging. Google Analytics just became a tad less useful. I still remember Google promising a single digit impact at most. Those days are long gone however. Recent estimates put the current overall impact between 32-37%. So between 32-37% of the search keywords – which people use to get to your site – will be hidden.
There is a whole website dedicated to tracking the percentage of not provided traffic: http://www.notprovidedcount.com/
I’m a lot more pessimistic, however. There are some websites I’ve worked with, which had close to 90% of their keyword data not provided. Currently, if you simply disregard your not provided keyword data, or secretly hope it will decrease then I have bad news for you.
You must deal with it
If you ever see this: https:// , then you are using a secure HTTPS protocol and you too probably don’t pass along data about your search keywords. HTTPS is HTTP (HyperText Transfer Protocol) plus SSL (Secure Socket Layer) but just look for https:// and you will know. If you look at the biggest sources of withheld keyword data, then you will clearly see, that it will only continue to increase.
- Logged in to your Google account? No keyword data!
- Coming from G+? No keyword data!
- Logged in at YouTube? No keyword data!
- Surfing at the office? Probably no keyword data either but it depends on how your office network is set up.
It’s not hard to see, that this is not likely to decrease.
On the flip side, Facebook graph search finally passes along keyword data. That is one less thing to worry about. I hope they keep it this way. Facebook really wants to attract advertisers. If you can believe internet rumors so do Google (for AdWords) and that’s why we’ve withheld keywords in the first place. I don’t know about that, but it is definitely an interesting theory.
If you have significant traffic from other search engines, then you can rest assured. This is only a Google-specific issue.
Google Analytics won’t help you
There are many misleading tutorials and blog posts out there claiming that through some esoteric trickery (incantations and possible animal sacrifice…) you can somehow extract the missing data. Wrong! Google Analytics won’t help you with this one.
I have even read one, which advised the reader on how to filter (not provided) keyword data out, so you can put it out of your mind. I hope you don’t do that! That’s about the worst thing you can do. There are some applications and techniques, which are able to help you.
Google Webmaster Tools
I personally like to use GWT very much, “GA’s neglected little brother”. Even though it has its flaws (quite a few actually: bucket data, rounding, slow, limited functionality, not frequently updated), it is still useful in many ways. I always connect a client’s GWT (and AdWords too for that matter) to it’s GA, so I won’t miss out on anything. GWT has its uses. Just think about sitemaps or the disavow tool. Now, it has a new functionality, which you can use. You can make an educated GUESS (notice the emphasis on guess) about the missing keywords.
You can find the data you need at Traffic > Search Queries. There are other things like keyword impression, clicks and click-trough rate and average position but I wouldn’t really put too much emphasis on them. Especially not on average position, as in my experience the likelihood of that to be accurate is the same as drinking too many beers and then gentlemanly discuss politics or sports. In short, not very likely.
To our relief all keywords are provided! If you check your Analytics data and compare it to GWT, then you can get a somewhat accurate picture of the missing keywords. However, I wouldn’t put this in a report for my boss or for a client. We need something more.
I’m in luck. Most of the Analytics accounts I handle have AdWords campaigns set up and running. Some are bigger, some are smaller and some are quite sporadic, but it doesn’t really matter. When it comes to paid search results, there are no withheld keywords. I always connect my AdWords to my GA and you should, too! If you have connected your AdWords, then you can find these keywords at Traffic Sources > Advertising > AdWords > Matched Search Queries.
This won’t be enough. At this point, you can guess the missing keywords for organic search results. And for paid results, we know everything. There is a little trick I use to get an even more accurate picture.
Branded or not branded
Usually, we think in terms of branded and not-branded keywords. Branded keywords (if your on-page is in order) mostly or exclusively point to your home page depending on the website in question.
To tell them apart you just have to look at the keyword landing pages. Here’s how:
Filter for Landing pages
Filter for (not provided)
…or without parenthesis.
Now you have a nice list of individual keywords and their landing pages. You can tell branded keywords from non-branded keywords by simply looking at results that went to the home page (marked with a “/”), which is probably a branded keyword. The others which went to your pages (“/example”) are non-branded keywords. If you repeat the process with the webmaster tools – which we discussed at the beginning – for your non-branded keywords, then you can now give a reliable list of keywords for non-paid, organic, non-branded search keywords and branded search keywords.
The whole process stripped-down to the bare minimum looks like this:
- GA (identifying percentage of not provided, outlining the problem)
- GWT (making an educated guess about the missing data based on your knowledge of the website and the available data and GWT search keyword data)
- Checking AdWords campaigns and paid search keywords
- Use GA to differentiate between branded and non-branded keywords
- Check GWT again and AdWords again for brand, non-branded, paid and organic separately using the right filters
- Create a structure for your data (excel table), compare your findings and make an educated guess
I’m a regular at many SEO forums, groups, websites and YouTube channels. I have encountered a thousand and one misconceptions about the missing keywords, the infamous (not provided). I want to set the most often encountered misconceptions straight. If you are an SEO or some other online marketing professional, then some of them probably seem trivial and self-explanatory.
- Google Analytics Premium still won’t provide you with the missing keywords
- Only Google withholds the organic search keywords, other search engines don’t
- Facebook graph search provides keywords
- Expensive custom analytics software still won’t give you the missing keywords
If you have followed the instructions and avoided the common misconceptions then you will get a much more accurate picture about the missing keywords. I would still not put this in a report but at least now you can do your job as an SEO.
“You can’t manage what you can’t measure.” may not be true in every case but it is definitely true for online marketing.
Stay tuned for my next article! One of my colleagues attended the latest Link Love conference and returned with really groundbreaking ideas. I still test them and research them but I will definitely share the results with you.