So This is my Python Code , After running this code when you reload the webdriver i face the error Access Denied You don't have permission to access site

I have also tried he Headless Method but that doesn't also work .

from time import sleep
import random
from selenium.webdriver.common.keys import Keys
import selenium
from selenium.webdriver.common.by import By
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.chrome.options import Options



class StockData:
    def __init__(self):
#         chrome_options = Options()
#         chrome_options.add_argument('user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36')
#         self.driver=webdriver.Chrome(executable_path=r'C:\Users\Arth\Desktop\codes\chromedriver.exe',chrome_options=chrome_options)
#         agent = self.driver.execute_script("return navigator.userAgent")
        options = Options()
        options.headless = True
        profile = webdriver.FirefoxProfile()
        profile.set_preference("general.useragent.override", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36")
        self.driver = webdriver.Firefox(profile,executable_path=r'C:\Users\Arth\Desktop\codes\geckodriver.exe')

        
    def goSite(self):
        driver=self.driver
        driver.get("https://www.nseindia.com/market-data/volume-gainers-spurts")
        sleep(3)
    def start(self):
        driver=self.driver
        #driver.find_element_by_xpath("//img[contains(@title, 'Refresh') ]").click()
        sleep(1)
        names=driver.find_elements_by_xpath("//a[contains(@title, '_blank') ]")
        print(names)
        
bot=StockData()
bot.goSite()
bot.start()```

CodePudding user response：

This error message...

Access Denied You don't have permission to access site

...implies that Selenium driven GeckoDriver initiated firefox Browsing Context is being detected as a BOT and further navigation is blocked.

Historically NSE India have been long known to be protected by Bot Manager an advanced bot detection service provided by Akamai and the response gets blocked.

You can find a couple of relevant detailed discussions in:

• Clicking on Get Data button for Monthly Settlement Statistics on nseindia.com doesn't fetch results using Selenium and Python
• Dynamic dropdown doesn't populate with auto suggestions on https://www.nseindia.com/ when values are passed using Selenium and Python

Solution

To conceal the fact that the WebDriver / GeckoDriver is automation driven you can follow the steps mentioned in the discussion How to Conceal WebDriver in Geckodriver from BotD in Java?

CodePudding user response：

You don't need selenium for what you are doing. Just use the requests library.

We first make a request to the main site, to get the cookies for our second request. Then we make our second request to the API.

request.Session() lets use store the cookies from the previous request.

You can see the request they make in the network tab, in your browser.

I've used their API Url, below.

import requests

session = requests.Session()
session.headers["User-Agent"] ="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:96.0) Gecko/20100101 Firefox/96.0"

session.get("https://www.nseindia.com/market-data/volume-gainers-spurts")

result = session.get("https://www.nseindia.com/api/live-analysis-volume-gainers")

print(result.json())

I've tested it and it should work.