Python code to grab KeywordDiscovery API data
Ian Lurie Nov 17 2010
If you use the KeywordDiscovery API, and Python, my pain is your gain. It took me a few hours to get this to work. You can grab it and go. Here’s the function, written in my usual Python Pigdin. I don’t recommend using it without a passing knowledge of Python, but that’s up to you:
def kwdiscovery(username,password,phraselist): base64string = base64.encodestring('%s:%s' % (username, password))[:-1] authheader = "Basic %s" % base64string apiurl = "http://api.keyworddiscovery.com/queries.php?queries=" separator = "%0D%0A" counter = 1 for phrase in phraselist: # make sure there's no funny characters try: phrase.decode('ascii') except UnicodeDecodeError: continue phrase = phrase.replace(" ","+") phrase = phrase.replace("n","") if (counter > 1): apiurl = apiurl + separator + phrase else: apiurl = apiurl + phrase counter = counter + 1 apiurl = apiurl + "&empty=1" req = urllib2.Request(apiurl) req.add_header("Authorization", authheader) blah = urllib2.urlopen(req) # because sometimes, things just go wrong try: result = ET.parse(blah) resultlist =  lst = result.findall("r") for item in lst: this = item.attrib["q"],item.attrib["m"] resultlist.append(this) except: this = "__one of the words in this request caused an error:",apiurl resultlist = [this] return resultlist
And here’s how you’d use the function:
#!/usr/bin/python import string import sys import httplib import urllib2 from urllib2 import Request, urlopen, URLError import xml.etree.ElementTree as ET import base64 f = open('longw.txt','r') g = open('words_countedlongtail.txt','w') words = f.readlines() username = "ENTER KEYWORDDISCOVERY USERNAME HERE" password = "ENTER KEYWORDDISCOVERY PASSWORD HERE" start = 0 count = len(words) while (count > 0): count = count - 9 end = start + 9 a = words[start:end] print "sent ",a resultlist = kwdiscovery(username,password,a) for l in resultlist: q = str(l) m = str(l) line = q + "t" + m + "n" g.write(line) print "received ",line start = end f.close() g.close()
Who knows, I might even create a web interface one of these days. In my spare time.
Related and recent
CEO & Founder
Ian Lurie is CEO and founder of Portent Inc. He’s recorded training for Lynda.com, writes regularly for the Portent Blog and has been published on AllThingsD, Forbes.com and TechCrunch.
Ian speaks at conferences around the world, including SearchLove, MozCon, SIC and ad:Tech. Follow him on Twitter at portentint. He also just published a book about strategy for services businesses: One Trick Ponies Get Shot, available on Kindle.