Spiders code


The from scrapy import Request
The from scrapy. Spiders import spiders
The from shetu_spider. Items import ShetuSpiderItem

The class image_download (spiders) :
Name='image'

Def start_requests (self) :
Url='http://699pic.com/photo/'
Yield Request (url)

Def parse (self, response) : # analytic function - image list page
Urls=response. Xpath ("//div [@ class='pl - the list] a [1]//@ href "). The extract ()
For I in range (len (urls) :
Yield Request (+ urls (' HTTP: '[I]), the callback=self. Parse_image)

Def parse_image (self, response) :
The item=ShetuSpiderItem ()

# get all pictures url address
Url=response. Xpath ("//li [@ class='list']/a/img/@ SRC "). The extract ()
If the url:
Title=the response. Xpath ("//li [@ class='list']/a/img/@ title "). The extract ()
The item [' title ']=title
Item [' image_urls] url=
Yield item
# for the next page address
Next_url=response. Xpath ("//a [@ class='downPage']/@ href "). The extract ()

If next_url:
Next_url=response. Urljoin (next_url [0])
Yield Request (next_url, callback=self parse_image)


Methods also took on the net, but still this error, please the great god, give some indicative opinions

CodePudding user response:

Problem solved, we have the same problem, please refer to my blog
https://blog.csdn.net/wenquan19960602/article/details/104860522