Ad

Python - List Index Out Of Range -

I found a script to Program to build a focused crawler in Python. This script stopped in function (google_scrape). in this function I've been getting error whenever executing it. This error is (List Index Out Of Range) Can you help me?

# Uses google search engine to find out the results for user query. 
def google_scrape(query):
    address = "http://www.google.com/search?q=%s&num=100&hl=en&start=0" % (urllib.quote_plus(query))
    request = urllib2.Request(address, None, {'User-Agent':'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11'})
    urlfile = urllib2.urlopen(request)   
    page = urlfile.read()
    soup = BeautifulSoup(page)
    links = []
    count = 0
    li = soup.findAll('li', attrs={'class':'g'})
    while count < 10:
        sLink = li[count].find('a')
        links.append(sLink['href']) 
        count += 1
    return links

Error Stack: enter image description here

Ad

Answer

Using The limit argument:

li = soup.findAll('li', attrs={'class':'g'}, limit=10)

for elem in li:
    sLink = elem.find('a')
    links.append(sLink['href'])
    count += 1
Ad
source: stackoverflow.com
Ad