Python helium get contents of table after click-CodePudding

I am using helium to scrape a webpage.

result = start_firefox(                                                                                   
    "https://www.medtronic.com/covidien/en-us/products/brain-monitoring/bis-monitoring-system.html",      
)                                                                                                         
                                                                                                          
result.find_element_by_class_name("js-open-table-overlay").click()

After the click action i am presented with a table and i need to scrape the contents of the table but How do i select the table after the click ?

CodePudding user response：

You would need to use find_elements_... to get all <table>, and use for-loop to work with every table separatelly amd use (nested) for-loop to get <tr> (row) and <th> (header) in table, and use (nested) for-loop to get <td> (cell) in row. And you would have to add elements to correct (nested) lists.

But this page uses normal <table> and this table is all time in HTML (not added by javaScript) so it can be simpler to use pandas.read_html() to get all <table>

import pandas as pd

url = "https://www.medtronic.com/covidien/en-us/products/brain-monitoring/bis-monitoring-system.html"

all_tables = pd.read_html(url)

for table in all_tables:
    print(table.to_string())

Result

     ORDER CODE                                                      DESCRIPTION UNIT OF MEASURE  QUANTITY
0      186-1014                        BIS™ Complete 4-Channel Monitoring System            Each         1
1      186-0210                                  BIS™ Complete 2-Channel Monitor            Each         1
2  186-0224-AMS  BIS™ LOC 4-Channel Monitor with Patient Interface Cable (PIC-4)            Each         1
3  186-0195-AMS          BIS™ LOC 2-Channel Monitor with Patient Interface Cable            Each         1
4      186-0212                                            BIS™ Bilateral Sensor            Each         1

              DISPLAY                                               BIS™ COMPLETE 2-CHANNEL MONITOR                                     BIS™ COMPLETE 4-CHANNEL MONITORING SYSTEM
0          Parameters                                                    BIS, SQI, EMG, SR, BC, EEG                                           BIS, SQI, EMG, SR, BC, TP, SEF, EEG
1  Trended parameters                                                         BIS, SQI, EMG, SR, BC                                                    BIS, SQI, EMG, SR, BC, SEF
2          BIS™ alarm  Upper and lower limit, results in visual and audible alert when out of range  Upper and lower limit, results in visual and audible alert when out of range