Home > Back-end >  How to implement Java similar python functions of the crawler
How to implement Java similar python functions of the crawler

Time:11-22

I want to download other sites in my Java code files, the problem is he gave the url must enter the user name and password to download, is there any way can be implemented in my code input user name password and the function of the download file

CodePudding user response:

Using httpClient simulation request after landing the cookies or directly to the user name password when parameters inside

CodePudding user response:

Crawler is actually using a component that is easy to implement, I use a few languages are realized,
Did not have another login authentication code, the slider,

CodePudding user response:

Okhttp3 can automatically manage the cookie

Jsoup can friendly parsing HTML

Shuangjian combination

CodePudding user response:

Java open source crawler frame
Crawler4j
WebMagic
  • Related