As title, take the contents of the website XHR need to climb, to find the url,

But the url can't open, shows that 404

The preview and the response has a json content, however, requires the content inside, consult each great god how python crawl?

CodePudding user response:

Add: get request

CodePudding user response:

It is ok to add a request header

CodePudding user response:

Thank you, I didn't think so important that I forgot the header

CodePudding user response:

The

reference flow> stock reply: 3/f

thank you very much, I didn't think so important that I forgot the header

There was a time, couldn't remember to add the content request header

CodePudding user response:

Who answered, XHR can't find the content inside how to climb

CodePudding user response:

How to get the preview of the content

CodePudding user response:

Yes, climb before, besides, it's important to know UA=the user-agent, also know other headers=request header, as well as more relevant logic

Can reference I don't understand:

The core of the crawler steps to crawl the data you need: the crawler technology



And understand the basic logic, crawler to choose corresponding language implementation

For example: Python

How to written in Python crawler

If you don't understand, you can refer to my finishing the actual examples:

Grab baidu how hot list, written in Python crawler