Warm tip: This article is reproduced from stackoverflow.com, please click
python scrapy xpath

Xpath returning same result

发布于 2020-03-27 10:21:55

Trying to use Xpath to scrape card name off the following website, https://www2.trollandtoad.com/buylist/?_ga=2.123753418.115346513.1562026676-1813285172.1559913561#!/M/10591, but it keeps returning the same result each time. I need it to output all the card names from that link, but it just gives me the same one over and over again.

def parse(self, response):
        #  Initialize item to function GameItem located in items.py, will be called multiple times
        item = GameItem()
        # Extract card category from URL using html code from website that identifies the category.  Will be outputted before rest of data
        for data in response.css('tr.ng-scope'):
            item["Set"] =data.css("a.ng-binding.ng-scope::text").get()
            if item["Set"] == None:
                item["Set"] = data.css("span.ng-binding.ng-scope::text").get()
            item["Card_Name"]  = data.xpath("//div/table/tbody/tr/td[contains(@class,'buylist_productname item')]/a/text()").get()

I tried using getall() but it would not work correctly either. It would give back all the card names but it would not be paired with the other data I scraped correctly. Instead of outputting one card name for one price and so on it would give me all the card names in one line along with the price for the first card and so on.

Questioner
Tom
Viewed
74
gangabass 2019-07-03 22:21

You need relative XPath:

item["Card_Name"]  = data.xpath(".//td[2]/a/text()").get()

UPDATE Fixed your XPath