I used cloudscraper to scrape this website oddschecker. I ran it locally on my computer, it works fine. But when I used a Digital Ocean VPS, Cloudflare denied my access, with an error message saying:

Access denied

This website is using a security service to protect itself from online attacks.

I'm not sure what that means - is Cloudflare blocking my VPS's IP address? Do I have to use a proxy to scrape it?

CodePudding user response：

Yes, this error you are seeing is due to a Cloudflare firewall rule that you are hitting. The Cloudflare firewall has a list of 20 different triggers to block/allow requests so it's hard to say exactly if is the IP trigger that's being blocked in this case but that's generally the case. See some examples here.

If you are not even able to access the site from the beginning using a new IP address/VPS, it's possible the trigger has to do with the behaviour of your request rather than the source - e.g. rate limiting (although the error message would be different in this case), number of requests per minute, method used to access, reputation of network block (ASN) etc.