Is there a loop that will iterate through the siblings elements, if it gets to one of the studentscreening(see below) and that student does not have the tag being used, then input null/na ??

Below is what is in my xml file [studentinfo.xml]:

<?xml version="1.0" encoding="UTF-8"?>
<StudentBreakdown>
<Studentdata>
    <StudentScreening>
        <name>Sam Davies</name>
        <age>15</age>
        <hair>Black</hair>
        <eyes>Blue</eyes>
        <grade>10</grade>
        <teacher>Draco Malfoy</teacher>
        <dorm>Innovation Hall</dorm>
    </StudentScreening>
    <StudentScreening>
        <name>Cassie Stone</name>
        <age>14</age>
        <hair>Science</hair>
        <grade>9</grade>
        <teacher>Luna Lovegood</teacher>
    </StudentScreening>
    <StudentScreening>
        <name>Derek Brandon</name>
        <age>17</age>
        <eyes>green</eyes>
        <teacher>Ron Weasley</teacher>
        <dorm>Hogtie Manor</dorm>
    </StudentScreening>
</Studentdata>
</StudentBreakdown>

My code is iterating through the studentinfo.xml file and inputting the information into a pandas dataframe(df1) per the columns I've mapped the tags to.

Below is a sample of my code:

import pandas as pd
from bs4 import BeautifulSoup
with open('studentinfo.xml', 'r') as f:
    file = f.read()  

def parse_xml(file):
    soup = BeautifulSoup(file, 'xml')
    df1 = pd.DataFrame(columns=['StudentName', 'Age', 'Hair', 'Eyes', 'Grade', 'Teacher', 'Dorm'])
    all_items = soup.find_all('info')
    items_length = len(all_items)
    for index, info in enumerate(all_items):
        StudentName = info.find('<name>').text
        Age = info.find('<age>').text
        Hair = info.find('<hair>').text
        Eyes = info.find('<eyes>').text
        Grade = info.find('<grade>').text
        Teacher = info.find('<teacher>').text
        Dorm = info.find('<dorm>').text
      row = {
            'StudentName': StudentName,
            'Age': Age,
            'Hair': Hair,
            'Eyes': Eyes,
            'Grade': Grade,
            'Teacher': Teacher,
            'Dorm': Dorm
        }
        
        df1 = df1.append(row, ingore_index=True)
        print(f'Appending row %s of %s' %(index 1, items_length))
    
    return df1  

When I try to run the code I get this error: 'AttributeError: 'NoneType' object has no attribute 'text'' Which my conclusion as to why I was getting this error was because not every StudentScreening has the same child tags being used.

What condition can be added to my code that says: " As I am looping through, If an element tag is not present, input null in the dataframe and continue to enumerate over the file" ??????

CodePudding user response：

While using pandas simply use its pandas.read_xml():

pd.read_xml(xml, xpath='.//StudentScreening')

Example

import pandas as pd

xml = '''
<StudentBreakdown>
<Studentdata>
    <StudentScreening>
        <name>Sam Davies</name>
        <age>15</age>
        <hair>Black</hair>
        <eyes>Blue</eyes>
        <grade>10</grade>
        <teacher>Draco Malfoy</teacher>
        <dorm>Innovation Hall</dorm>
    </StudentScreening>
    <StudentScreening>
        <name>Cassie Stone</name>
        <age>14</age>
        <hair>Science</hair>
        <grade>9</grade>
        <teacher>Luna Lovegood</teacher>
    </StudentScreening>
    <StudentScreening>
        <name>Derek Brandon</name>
        <age>17</age>
        <eyes>green</eyes>
        <teacher>Ron Weasley</teacher>
        <dorm>Hogtie Manor</dorm>
    </StudentScreening>
</Studentdata>
</StudentBreakdown>'''

pd.read_xml(xml, xpath='.//StudentScreening')

Output

CodePudding user response：

You can iterate over your xml file with ElementTree to create a list of dictionaries that you'll then convert to a dataframe:

import pandas as pd
import xml.etree.ElementTree as ET

tree = ET.parse('studentinfo.xml')
root = tree.getroot()

arr = []

for student_screening in root.iterfind('.//StudentScreening'):
    arr.append({el.tag: el.text for el in student_screening})

df = pd.DataFrame(arr)
print(df)

Output:

            name age     hair   eyes grade        teacher             dorm
0     Sam Davies  15    Black   Blue    10   Draco Malfoy  Innovation Hall
1   Cassie Stone  14  Science    NaN     9  Luna Lovegood              NaN
2  Derek Brandon  17      NaN  green   NaN    Ron Weasley     Hogtie Manor

CodePudding user response：

Try:

import pandas as pd
from bs4 import BeautifulSoup

html_doc = """\
<?xml version="1.0" encoding="UTF-8"?>
<StudentBreakdown>
<Studentdata>
    <StudentScreening>
        <name>Sam Davies</name>
        <age>15</age>
        <hair>Black</hair>
        <eyes>Blue</eyes>
        <grade>10</grade>
        <teacher>Draco Malfoy</teacher>
        <dorm>Innovation Hall</dorm>
    </StudentScreening>
    <StudentScreening>
        <name>Cassie Stone</name>
        <age>14</age>
        <hair>Science</hair>
        <grade>9</grade>
        <teacher>Luna Lovegood</teacher>
    </StudentScreening>
    <StudentScreening>
        <name>Derek Brandon</name>
        <age>17</age>
        <eyes>green</eyes>
        <teacher>Ron Weasley</teacher>
        <dorm>Hogtie Manor</dorm>
    </StudentScreening>
</Studentdata>
</StudentBreakdown>"""


soup = BeautifulSoup(html_doc, "xml")

all_data = []
for s in soup.select("StudentScreening"):
    all_data.append(
        {
            "name": s.find("name"),
            "age": s.age,
            "eyes": s.eyes,
            "grade": s.grade,
            "teacher": s.teacher,
            "dorm": s.dorm,
        }
    )

df = pd.DataFrame(all_data).apply(lambda x: [v.text if v else "N/A" for v in x])
print(df)

Prints:

            name age   eyes grade        teacher             dorm
0     Sam Davies  15   Blue    10   Draco Malfoy  Innovation Hall
1   Cassie Stone  14    N/A     9  Luna Lovegood              N/A
2  Derek Brandon  17  green   N/A    Ron Weasley     Hogtie Manor