Home > other >  The selenium fetching problem
The selenium fetching problem

Time:11-25

Open directly on a link in the Chrome is no problem, but if I by browser=webdriver. Chrome (), then the get (url) to open, and then at 403, why? The selenium webdriver. Chrome () is not adjustable Chrome directly open the corresponding web page

CodePudding user response:

First get (' www.baidu.com ') to see if normal, if normal, there to see no difference between HTTP, HTTPS, in selenium to open the browser to walk again target site access to the process


The selenium webdriver. Chrome () is called by chromedriver Chrome, remote is not directly open,

CodePudding user response:

To what's wrong? Guide the package? The source code to look at it?

CodePudding user response:

The following PaoFan is 403, if the website on the browser open and no problems
 
Def chromeBrowser () :
# to create chrome parameter object
Opt=webdriver. ChromeOptions ()
Opt. Add_argument (
'the user-agent=Mozilla/5.0 (Windows NT 10.0; Win64. X64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36 ')

Opt. Add_argument (' - no - the sandbox) # solve DevToolsActivePort file does not exist error

Opt. Add_argument (' window - size=1920 x3000 ') # specified browser resolution
Opt. Add_argument (' - disable - gpu ') # Google documents mentioned the need to add this property to avoid a bug

# opt. Add_argument (' - hide - scrollbars') # to hide the scroll bar, deal with some special page
# opt. Add_argument (' blink - Settings=imagesEnabled=false ') # loading pictures, not speed
# opt. Add_argument (' - headless ') # browser does not provide visual page. If the system does not support under Linux visualization will start without the failure
# to create chrome object the interface
Browser=webdriver. Chrome ()
The maximize_window ()
The get (' http://www.iwencai.com/diag/block-detail? CodeType=stock& Codes=603501 & amp; Info=% 7 b % 22 view % 22% % 3 a 7 b % 22 nolazy % 22% 3 a1%2 c % 22 parsearr % 22% % 3 a 7 b % 22 _v % 22% % 22 new 3 a 22% % 22% 2 c % 22 daterange % 3 a 5 b % 2220190101% % 22% 2 c % 2220191231% 22% 5 d % 2 c % 22 staying % 22% % 3 a 5 b % 5 d % 2 c % 22 querycompare % 22% % 3 a 5 b % 5 d % 2 c % 22 comparesofindex % 22% % 3 a 5 b % 5 d % d % d % 7 July 7 d & amp; Logid=& amp; Pid=6935 ')

The implicitly_wait (50)
F=open (' HTML. TXT ', 'w', encoding="utf-8") # need to develop file encoding
F.w rite (browser. Page_source. Encode (' utf-8). Decode ())
F. lose ()
The quit ()
  • Related