Home > Net >  Python scraping HTTPError: 403 Client Error: Forbidden for url:
Python scraping HTTPError: 403 Client Error: Forbidden for url:

Time:11-30

My python code used to work, but when I tried it today it did not work anymore. I assume the website owner forbade non browsers requests recently.

code

import requests, bs4
res = requests.get('https://manga1001.com/日常-raw-free/')
res.raise_for_status()
print(res.text)

I read that adding header in the requests.get method may work, but I don't know which header info exactly I need to make it work.

error

---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
<ipython-input-15-ed1948d83d51> in <module>
      3 # res = requests.get('https://manga1001.com/日常-raw-free/', headers=headers_dic)
      4 res = requests.get('https://manga1001.com/日常-raw-free/')
----> 5 res.raise_for_status()
      6 print(res.text)
      7 

~/opt/anaconda3/lib/python3.8/site-packages/requests/models.py in raise_for_status(self)
    939 
    940         if http_error_msg:
--> 941             raise HTTPError(http_error_msg, response=self)
    942 
    943     def close(self):

HTTPError: 403 Client Error: Forbidden for url: https://manga1001.com/           
  • Related