Scrapy error : 'NoneType' object has no attribute 'strip'

Question

I am using scrapy 1.4,the version of twisted is 17.5.0,python is 3.6.3.

html is like this：

<span class="number"> 20</span>

when I run the spider,there is an error:

item['number'] = response.xpath('//span[@class="number"]/text()').extract_first().strip() AttributeError: 'NoneType' object has no attribute 'strip'

what should I do?

Your extract_first() function most likely returns a None element in this specific case. Either validate what you return in the function code in a try/except block and exit there if you can't get a value you can pass to another function, or validate the answer before assignment so your strip() doesn't error out on unexpected None typed objects. I would do it before assignment as maybe you want extract_first to return None sometimes. — BoboDarph
– BoboDarph, Commented Nov 2, 2017 at 12:12

peregrino · Accepted Answer · 2017-11-02 10:32:09Z

Seems like problem is in the xpath that you wrote or in the document itself. extract_first method returns None if no match found. I've checked your example in online xpath validators and seems like everything is fine

Collectives™ on Stack Overflow

Scrapy error : 'NoneType' object has no attribute 'strip'

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related