Always getting a 403 response from curseforge.com (python)-CodePudding

main.py

import requests

link = 'https://www.curseforge.com/minecraft/mc-mods/jei'
header = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36'
    }

req = requests.get(link, headers=header)
print(req)

I have tried adding the 'referer' field but it still gives me a 403 response, I even tried using requests.Session() but that didn't work too, maybe the server needs some specific field to accept the request? Any help is appreciated!

CodePudding user response：

Because that websites uses captcha to prevent non-humans to access the website. You can see the captcha page by saving the result of the request in a file and then opening it in a browser:

from requests import get

result = get("https://www.curseforge.com/minecraft/mc-mods/jei")

with open("temp.html", "w") as f:
    f.write(result.text)

CodePudding user response：

Fixed it by adding this to the url

'http://webcache.googleusercontent.com/search?q=cache:'