Get values from different tags using a condition in BeautifulSoup-CodePudding

I have a xml file like this:

data =   """<entity type="protein" entityId="A">
    <segment segId="7ddd_A_1_1208" start="1" end="1208">
      <listResidue>
        <residue dbSource="PDBe" dbCoordSys="PDBe" dbResNum="1" dbResName="MET">
          <crossRefDb dbSource="PDB" dbCoordSys="PDBresnum" dbAccessionId="7ddd" dbResNum="null" dbResName="MET" dbChainId="A"/>
          <crossRefDb dbSource="UniProt" dbCoordSys="UniProt" dbAccessionId="P0DTC2" dbResNum="1" dbResName="M"/>
          <crossRefDb dbSource="NCBI" dbCoordSys="UniProt" dbAccessionId="2697049" dbResNum="1" dbResName="M"/>
          <residueDetail dbSource="PDBe" property="Annotation">Not_Observed</residueDetail>
        </residue>
        <residue dbSource="PDBe" dbCoordSys="PDBe" dbResNum="14" dbResName="GLN">
          <crossRefDb dbSource="PDB" dbCoordSys="PDBresnum" dbAccessionId="7ddd" dbResNum="14" dbResName="GLN" dbChainId="A"/>
          <crossRefDb dbSource="UniProt" dbCoordSys="UniProt" dbAccessionId="P0DTC2" dbResNum="13" dbResName="Q"/>
          <crossRefDb dbSource="NCBI" dbCoordSys="UniProt" dbAccessionId="2697049" dbResNum="14" dbResName="Q"/>
          <residueDetail dbSource="PDBe" property="codeSecondaryStructure">T</residueDetail>
          <residueDetail dbSource="PDBe" property="nameSecondaryStructure">loop</residueDetail>
        </residue>
  </entity>
  <entity type="protein" entityId="B">
    <segment segId="7ddd_B_1_1208" start="1" end="1208">
      <listResidue>
        <residue dbSource="PDBe" dbCoordSys="PDBe" dbResNum="1" dbResName="MET">
          <crossRefDb dbSource="PDB" dbCoordSys="PDBresnum" dbAccessionId="7ddd" dbResNum="1" dbResName="MET" dbChainId="B"/>
          <crossRefDb dbSource="UniProt" dbCoordSys="UniProt" dbAccessionId="PXCDT" dbResNum="1" dbResName="M"/>
          <crossRefDb dbSource="NCBI" dbCoordSys="UniProt" dbAccessionId="2697049" dbResNum="1" dbResName="M"/>
          <residueDetail dbSource="PDBe" property="Annotation">Not_Observed</residueDetail>
        </residue>
        <residue dbSource="PDBe" dbCoordSys="PDBe" dbResNum="16" dbResName="VAL">
          <crossRefDb dbSource="PDB" dbCoordSys="PDBresnum" dbAccessionId="7ddd" dbResNum="16" dbResName="VAL" dbChainId="B"/>
          <crossRefDb dbSource="UniProt" dbCoordSys="UniProt" dbAccessionId="P0DTC2" dbResNum="16" dbResName="V"/>
          <crossRefDb dbSource="NCBI" dbCoordSys="UniProt" dbAccessionId="2697049" dbResNum="16" dbResName="V"/>
          <residueDetail dbSource="PDBe" property="codeSecondaryStructure">T</residueDetail>
          <residueDetail dbSource="PDBe" property="nameSecondaryStructure">loop</residueDetail>
        </residue>"""

What I am trying to do is retrieve the dbResNum value from the same line as dbSource="PDB" and dbSource="UniProt". But, I want those values only if it doesn't have dbResNum="null" in the dbSource="PDB" line and it doesn't have any different value than dbAccessionId="P0DTC2" in the dbSource="UniProt" line. So, my output would be something like this:

PDB_Res = '13', '16' Uniprot_Res = '14, '16'

My code for trying to do that:

from bs4 import BeautifulSoup
import re
def not_null(dbresnum):
    return dbresnum and not re.compile("null").search(dbresnum)

xml_file = BeautifulSoup(data, 'lxml')
XML_without_null = xml_file.find_all(dbresnum=not_null)
for cross in XML_without_null:
    if cross('crossrefdb', {"dbaccessionid":"P0DTC2"}):
        Uniprot_id = cross['dbresnum']
    if cross('crossrefdb', {"dbsource":"PDB"}):
        PDB_id = cross['dbresnum']

The problem is the function not_null is not working properly. Any idea is welcome.

CodePudding user response：

First, need to change the parser from "lxml" to "xml".

Also note the XML snippet in your question is an invalid XML file with no closing segment or listResidue elements. There is no root XML element, only 2 entity elements which is not valid. BeautifulSoup handles invalid documents but always recommended to start with a valid XML document if possible.

If want to skip an entire residue group and all crossRefDb children in it then need to iterate over all residue and check if any child crossRefDb has has a null in dbresnum in the same line as dbSource="PDB".

Try something like this:

from bs4 import BeautifulSoup

data = '''<entity ...'''

soup = BeautifulSoup(data, 'xml')
uni_res = set()
pdb_res = set()
for residue in soup.find_all('residue'):
    if residue.find('crossRefDb', {'dbSource': 'PDB', 'dbResNum': 'null'}):
        continue
    for cross in residue.find_all('crossRefDb'):
        res_num = cross.get('dbResNum')
        if cross.get("dbAccessionId") == "P0DTC2":
            uni_res.add(res_num)
        if cross.get("dbSource") == "PDB":
            pdb_res.add(res_num)

print("PDB_Res =", pdb_res, "Uniprot =", uni_res)

Output:

PDB_Res = {'1', '16', '14'} Uniprot_Res = {'16', '13'}