Home > other >  Want to crawl your shopping cart information, but add login and without authentication login returns
Want to crawl your shopping cart information, but add login and without authentication login returns

Time:12-02

The from urllib. Request the import HTTPPasswordMgrWithDefaultRealm HTTPBasicAuthHandler, build_opener
The from urllib. Error import URLError
The import urllib
The from bs4 import BeautifulSoup
The username='here put user name'
Password='fill password here'
Url='here to put a shopping cart url (to log in to enter)'
P=HTTPPasswordMgrWithDefaultRealm (#) create a HTTPPasswordMgrWithDefaultRealm object
P.a dd_password (None, url, username, password) # add user name and password to this object
Auth_handler=HTTPBasicAuthHandler # (p) create a HTTPBasicAuthHandler instance, parameters for HTTPPasswordMgrWithDefaultRealm object
Opener=build_opener (auth_handler) # building a opener

Try:
Result=opener. Open (url)
HTML=result. The read (). The decode (' utf-8 'errors=' ignore ')
Soup=BeautifulSoup (HTML, 'LXML')
Things_need=Soup. Select (' # J_Item_1802600522449 & gt; Ul: NTH - child (1) & gt; Li: the NTH - child (6) & gt; Div: NTH - child (1) & gt; Em: NTH - child (1) ')
Print (things_need [0]. Get_text ()) #!!!!!!!!!! Here failed
Except URLError as e:
Print (" e.r eason)