how to skip the first table, and skip the second table head during parsing a local html file in pyth-CodePudding

I am trying to parse a local html file, I don't know why same codes resulted differently between sample html text and the whole html file. Can anyone help? I really appreciate it. The sample html text:

s = '''
<table width=90%>
   <tr>
      <td align="center" width=18%></td>
      <td align="left" width=15%></td>
   </tr>
</table>
<table border>
   <tr>
      <td nowrap="nowrap"><b>Rec</b></td>
      <td align="RIGHT" nowrap="nowrap"><b>ID</b></td>
   </tr>
   <tr>
      <td align="RIGHT" nowrap="nowrap" VALIGN=TOP>1</td>
      <td nowrap="nowrap" VALIGN=TOP><a href="smthing?DID=ID">ID<br />100100</a></td>
   </tr>
</table>
<p>
<style type="text/css">
 .....
</style>
<table border>
   <tr>
      <td nowrap="nowrap"><b>Rec</b></td>
      <td align="RIGHT" nowrap="nowrap"><b>ID</b></td>
   </tr>
   <tr>
      <td align="RIGHT" nowrap="nowrap" VALIGN=TOP>2</td>
      <td nowrap="nowrap" VALIGN=TOP><a href="smthing?DID=ID">ID<br />101101</a></td>
   </tr>
</table>
 '''

I have tried following:

''''

# with open('myfile.html', 'r', encoding='utf-8') as f: # when use the whole file 
# s = f.read() # when use the whole file 
    soup = BeautifulSoup(s, "html.parser")
    tables = [
        [
            [td.get_text(strip=True) for td in tr.find_all('td')]
            for tr in table.find_all('tr')
        ]
        for table in soup.find_all('table')
    ]
    table_data = [i.text for i in soup.find_all('td')]

    print(table_data)

'''' expected output:

Rec   ID
1     ID100100
2     ID101101

the current output is:

['', '', 'Rec', 'ID', '1', 'ID100100', 'Rec', 'ID', '2', 'ID101101']

also, when I implemented the same codes with the whole HTML file, the result was included something like below, did I miss something here:

'', '</tr>', '', '</table>', '', '</table>', '', '</center>', '', '<hr />', '', '<center>', '',

CodePudding user response：

You can apply list slicing

from bs4 import BeautifulSoup

s = '''
<table width=90%>
   <tr>
      <td align="center" width=18%></td>
      <td align="left" width=15%></td>
   </tr>
</table>
<table border>
   <tr>
      <td nowrap="nowrap"><b>Rec</b></td>
      <td align="RIGHT" nowrap="nowrap"><b>ID</b></td>
   </tr>
   <tr>
      <td align="RIGHT" nowrap="nowrap" VALIGN=TOP>1</td>
      <td nowrap="nowrap" VALIGN=TOP><a href="smthing?DID=ID">ID<br />100100</a></td>
   </tr>
</table>
<p>
<style type="text/css">
 .....
</style>
<table border>
   <tr>
      <td nowrap="nowrap"><b>Rec</b></td>
      <td align="RIGHT" nowrap="nowrap"><b>ID</b></td>
   </tr>
   <tr>
      <td align="RIGHT" nowrap="nowrap" VALIGN=TOP>2</td>
      <td nowrap="nowrap" VALIGN=TOP><a href="smthing?DID=ID">ID<br />101101</a></td>
   </tr>
</table>
 '''


 
soup = BeautifulSoup(s, "html.parser")
  
table = soup.find_all('table')[2]
#print(len(table))

data=[]
table_data = [i.text for i in soup.find_all('td')]

rec=table_data[-3]
num_1= table_data[-5]
num_2= table_data[-1]

data.append([rec,num_1,num_2])
print(data)

Output:

[['ID', 'ID100100', 'ID101101']]