Not Getting All The A Elements From Div Class Using Xpath And Scrapy

I have been trying to get all the properties from this website. When I access all of them on the main search page I can retrieve all the information from all the properties, however when I need the information from actual property link, it only seems to go through one property link.

The main issue is in the link part, so when I actually try to access the link of the property. I only get the link and information from the first property but not from all the others.

class PropDataSpider(scrapy.Spider):
    name = "remax"
    start_urls = ['']

    def parse(self, response):

        propertes = response.xpath("//div[@class='w-container main-content remodal-bg']")
        for prop in propertes:
            link = '' + prop.xpath("./a/@href").extract_first()
            agency =
            title = prop.xpath(
            price = prop.xpath(


           yield scrapy.Request(
                    'agency': agency,
                    'title': title,
                    'price': price,
                    'description': description,
                    'bedrooms': bedrooms,
                    'bathrooms': bathrooms,
                    'garages': garages,

 def parse_property(self, response):
        agency = response.meta["agency"]
        title = response.meta["title"]
        price = response.meta["price"]
        description = response.meta["description"]
        bedrooms = response.meta["bedrooms"]
        bathrooms = response.meta["bathrooms"]
        garages = response.meta["garages"]

        yield {'agency': agency, 'title': title, 'price': price, "description": description, 'bedrooms': bedrooms,'bathrooms': bathrooms, 'garages': garages}

What I would like to get is all the other links to properties. I am not sure what I am doing wrong and how to fix this.

Thank you very much for help!



You need couple of changes:

properties = response.xpath("//div[@class='w-container main-content remodal-bg']/a")
for prop in properties:
    link = '' + prop.xpath("./@href").extract_first()