Why use the IP pool douban crawler change IP can't contact with abnormal state?-CodePudding

Python, requests the library to use, to get IP from free IP pool, forge the user_agent, is set to the server after feedback abnormal change IP and user_agent, but still was identified by the server after the change is abnormal IP, combined with my own true IP open the page with a browser is asked to log in,

CodePudding user response:

Could you these agents have been sealed

CodePudding user response:

The

reference 1/f, archaeologists lx response:

may you these agents have been sealed

Don't feel like, every time the first IP is no problem, but once the trigger after the login prompt, in article how many IP can't, I suspect douban server's climb system to track the search request, refused to out after an unusually IP, made a similar search request will take a short time the IP as the same source, disable off together,

CodePudding user response:

reference period, on the second floor response:

Quote: reference 1/f, archaeologists lx response:
may you these agents have been sealed

Don't feel like, every time the first IP is no problem, but once the trigger after the login prompt, in article how many IP can't, I suspect douban server's climb system to track the search request, refused to out after an unusually IP, the IP in a short period of time made a similar search requests that are the same source, disabled,

Not so smart banIP strategy, as long as your Ip and Ua is dynamic,

As you say this kind of situation, on the successful use of your agent?

CodePudding user response:

I read your screenshot is the agent in the HTTP, watercress links should be HTTPS,

Probably no success using a proxy

Suggest you change agent IP to the HTTPS proxy,

CodePudding user response:

reference archaeologists lx reply: 3/f

Quote: refer to the second floor is not issue, reply:

Quote: reference 1/f, archaeologists lx response:
may you these agents have been sealed

Don't feel like, every time the first IP is no problem, but once the trigger after the login prompt, in article how many IP can't, I suspect douban server's climb system to track the search request, refused to out after an unusually IP, the IP in a short period of time made a similar search requests that are the same source, disabled,

Not so smart banIP strategy, as long as your Ip and Ua is dynamic,

As you say this kind of situation, on the successful use of your agent?

Problem solved, final inspection found out is to generate the IP code is wrong, has been with the same IP,,,