While webscrapping this error shows Not Acceptable! An appropriate representation of the requested r-CodePudding

I am trying to scrape data from a website but it shows this error. I don't know how to fix this.

b'<head><title>Not Acceptable!</title></head><body><h1>Not Acceptable!</h1><p>An appropriate representation of the requested resource could not be found on this server. This error was generated by Mod_Security.</p></body></html>'

This is my code

from bs4 import BeautifulSoup
import requests
import pandas as pd

url = 'https://insights.blackcoffer.com/how-is-login-logout-time-tracking-for-employees-in-office-done-by-ai/'

page = requests.get(url).content
page

Output

CodePudding user response：

You need to add user-agent and it works.
If you do not put user-agent of some browser, the site thinks that you are bot and block you.

from bs4 import BeautifulSoup
import requests

url = 'https://insights.blackcoffer.com/how-is-login-logout-time-tracking-for-employees-in-office-done-by-ai/'
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36"}
page = requests.get(url, headers=headers).content
print(page)