so the page I'm trying to use Web Scraping on is Private. It uses two-way authentication, which will not let me open the link through selenium. When I open the page manually I'm not asked for extra authentication.

The Page is self uses the same classes for all the tables on the page and the classes in the td tag are all mostly the same as well.

Here is the Table with the data I wan't to extract

Here is another Table on the same Page, which I don't need, but mostly has the same classes and tags

It really kills me that no other attributes or anything was added in order to make this a bit more simple. Since that is no the case, I'm really clueless how to continue to get the data.

Really open for any ideas. Thx in advance

CodePudding user response：

First, if the data are always in the same order, you can try to use a css selector, like driver.find_element(By.CSS_SELECTOR, "tr > td:nth-child(3)") to get the third td in the first tr for exemple.

If it don't work, and your goal is to get information related to a key in the table, you can make a loop to collect all the data from the table into a dictionnary, and then call the key that you want.