Selenium Unable to process further after executing script-CodePudding

This question is actually continued from here I have following link I got help from community regarding using script in selenium but main part here is I want to collect atleast 2-3 news and store in some json which can further be manipulated. I am not able to click the get those news section which is in bottom part of page any way I can scrape those news at least 1-2 new?

   stocks_data=["AKPL","NICA"]
   for stock_data in stocks_data:
            self.driver.get(f'https://merolagani.com/CompanyDetail.aspx?symbol={stock_data[0].lower()}')
            self.driver.execute_script(
                "document.getElementById('ctl00_ContentPlaceHolder1_CompanyDetail1_lnkNewsTab').click()")

CodePudding user response：

Use stocks_data=["AKPL","NICA"] instead of stocks_data="AKPL","NICA"]

CodePudding user response：

As for the multiple news, just use a loop over document.querySelectorAll('[id*="_lnkNewsTab"]') or some similar selector if that one is too wide.

As for the json, try to populate some imaginary div (that you can create in Javascript by document.body.innerHTML = '<div id="my_json"></div>') with the content of the elements you need (document.getElementById('my_json').innerHTML = document.getElementById('the id of the element you want the data from').innerHTML;) and finally get that div from python (see How to get text with Selenium WebDriver in Python)

Then you can manipulate however you like in python.

Something like:

   stocks_data=["AKPL","NICA"]
   for stock_data in stocks_data:
            self.driver.get(f'https://merolagani.com/CompanyDetail.aspx?symbol={stock_data[0].lower()}')
            self.driver.execute_script("document.getElementById('ctl00_ContentPlaceHolder1_CompanyDetail1_lnkNewsTab').click()")
            self.driver.execute_script("""
document.body.innerHTML  = '<div id="my_json"></div>';
var x = document.querySelectorAll('[id*="_lnkNewsTab"]');
x.map(v => {
  document.getElementById('my_json').innerHTML  = v.innerHTML   '--DELIMITER--'
});
            """)
            content = self.driver.find_element_by_id("my_json")
            content = content.text.split('--DELIMITER--')[:-1]