Ad

Scrapy Crawler What Tells Me This Output?

- 1 answer

My scrapy test code is the following. I tested it via scrapy shell and it worked. But now if I finally started to write a script no output appears. Is there something wrong? Thanks.

my code so far:

import scrapy


class CryptohunterSpider(scrapy.Spider):
    name = "cryptohunter"
    start_urls=["https://www.coingecko.com/de"]



def parse(self, response):

    for x in response.xpath('//div[@class="coin-content center"]/a/span/text()'):
        print (x.extract())

Output: enter image description here Sorry I couldn't input the shell, so I started a screenshot.

So I'm not sure if it's the error or not. If it should be an error, how show I handle it?

enter image description here How to get the full number out of this? It's always rounded by 2. Output: 0,07 instead of 0.0688852511227967

enter image description here

Ad

Answer

Indentation in Python is very important (more info about indentation).

Incorrect indentation can cause errors or wrong program execution. In your case parse() method does not belong to your class. So during execution scrapy tries to find parse() method in CryptohunterSpider and fails with error:

raise NotImplementedError('{}.parse callback is not defined'.format(self.__class__.__name__))

You need to properly indent your code so parse() belongs to the class

import scrapy

class CryptohunterSpider(scrapy.Spider):
    name = "cryptohunter"
    start_urls=["https://www.coingecko.com/de"]

    def parse(self, response):
        for x in response.xpath('//div[@class="coin-content center"]/a/span/text()'):
            print ("Extract: ", x.extract())
Ad
source: stackoverflow.com
Ad