Home > other >  Scrapy crawl dynamic pages, this is why the json data
Scrapy crawl dynamic pages, this is why the json data

Time:09-23

 def start_requests (self) : 
Url="https://v.taobao.com/micromission/req/selectCreatorV3.do? CateType=704 & amp; & _output_charset=utf-8 & amp; _input_charset=utf-8 & amp; _csrf=cea7ed4b business data catalog - 171 - b - 4-9 a0a - baf4b29f7730 '
Requests=[]
For I in range (2, 3) :
My_data=https://bbs.csdn.net/topics/{' cateType ':' 704 ', 'currentPage: STR (I),' _output_charset ':' utf-8 ', '_input_charset' : 'utf-8', '_csrf' : 'business data catalog cea7ed4b - 171 - b - 4-9 a0a - baf4b29f7730'}
Request=request (url, method="GET", the callback=self. The parse, body=json. The dumps (my_data), encoding="utf-8")
Requests. Append (request)
Return requests

Def parse (self, response) :
JsonBody=json. Loads (response. Body)


Here is the code, the json is the web link below


Feel less online about a get request for instance, could you tell me how to get to this page under the json data
This is should get into the
  • Related