Home > database >  Python loop through ndjson file data
Python loop through ndjson file data

Time:02-22

I have ndjson file Patients.ndjson

{"resourceType":"Patient","id":"ewnMwMK-UNvVvM.bakFSlkw3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/CodeSystem/v3-NullFlavor","code":"UNK","display":"Unknown"},"url":"ombCategory"},{"valueString":"Unknown","url":"text"}],"url":"http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"},{"extension":[{"valueString":"Unknown","url":"text"}],"url":"http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"},{"valueCodeableConcept":{"coding":[{"system":"urn:oid:1.2.840.114350.1.13.520.3.7.10.698084.130.657370.334258","code":"female"}]},"url":"http://open.epic.com/FHIR/StructureDefinition/extension/legal-sex"},{"valueCodeableConcept":{"coding":[{"system":"urn:oid:1.2.840.114350.1.13.520.3.7.10.698084.130.657370.334258","code":"female"}]},"url":"http://open.epic.com/FHIR/StructureDefinition/extension/sex-for-clinical-use"},{"valueCode":"F","url":"http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex"}],"identifier":[{"use":"usual","type":{"text":"EPIC"},"system":"urn:oid:1.2.840.114350.1.13.520.3.7.5.737384.0","value":"E9254"},{"use":"usual","type":{"text":"EXTERNAL"},"system":"urn:oid:1.2.840.114350.1.13.520.3.7.2.698084","value":"Z11363"},{"use":"usual","type":{"text":"FHIR"},"system":"http://open.epic.com/FHIR/StructureDefinition/patient-dstu2-fhir-id","value":"TYO3ktvhYAUhbae7JuBwDdpyIbUZc8kZG.bMW2ZwVnwgB"},{"use":"usual","type":{"text":"FHIR STU3"},"system":"http://open.epic.com/FHIR/StructureDefinition/patient-fhir-id","value":"ewnMwMK-UNvVvM.bakFSlkw3"},{"use":"usual","type":{"text":"PAS"},"system":"urn:oid:2.16.840.1.113883.2.1.3.12.1.1","value":"2005294"},{"use":"usual","type":{"text":"INTERNAL"},"system":"urn:oid:1.2.840.114350.1.13.520.3.7.2.698084","value":"    Z11363"}],"active":true,"name":[{"extension":[{"valueCode":"NL4","url":"http://hl7.org/fhir/StructureDefinition/humanname-assembly-order"}],"use":"official","text":"Test ADVANCEPREPTWO","family":"ADVANCEPREPTWO","given":["Test"],"_family":{"extension":[{"valueString":"Advancepreptwo","url":"http://hl7.org/fhir/StructureDefinition/humanname-own-name"}]}},{"extension":[{"valueCode":"NL4","url":"http://hl7.org/fhir/StructureDefinition/humanname-assembly-order"}],"use":"usual","text":"Test ADVANCEPREPTWO","family":"ADVANCEPREPTWO","given":["Test"],"_family":{"extension":[{"valueString":"Advancepreptwo","url":"http://hl7.org/fhir/StructureDefinition/humanname-own-name"}]}}],"gender":"female","birthDate":"2006-10-07","deceasedBoolean":false,"maritalStatus":{"text":"Single"}}
{"resourceType":"Patient","id":"exU8JSL0p8npSw5g1QYAyOw3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/CodeSystem/v3-NullFlavor","code":"UNK","display":"Unknown"},"url":"ombCategory"},{"valueString":"Unknown","url":"text"}],"url":"http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"},{"extension":[{"valueString":"Unknown","url":"text"}],"url":"http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"},{"valueCodeableConcept":{"coding":[{"system":"urn:oid:1.2.840.114350.1.13.520.3.7.10.698084.130.657370.334258","code":"female"}]},"url":"http://open.epic.com/FHIR/StructureDefinition/extension/legal-sex"},{"valueCodeableConcept":{"coding":[{"system":"urn:oid:1.2.840.114350.1.13.520.3.7.10.698084.130.657370.334258","code":"female"}]},"url":"http://open.epic.com/FHIR/StructureDefinition/extension/sex-for-clinical-use"},{"valueCode":"F","url":"http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex"}],"identifier":[{"use":"usual","type":{"text":"EPIC"},"system":"urn:oid:1.2.840.114350.1.13.520.3.7.5.737384.0","value":"E9340"},{"use":"usual","type":{"text":"EXTERNAL"},"system":"urn:oid:1.2.840.114350.1.13.520.3.7.2.698084","value":"Z11449"},{"use":"usual","type":{"text":"FHIR"},"system":"http://open.epic.com/FHIR/StructureDefinition/patient-dstu2-fhir-id","value":"TwzgMHMPt5OhnHkFA2H2DudL3FU8qcRZkZyn0F3extkAB"},{"use":"usual","type":{"text":"FHIR STU3"},"system":"http://open.epic.com/FHIR/StructureDefinition/patient-fhir-id","value":"exU8JSL0p8npSw5g1QYAyOw3"},{"use":"usual","type":{"text":"PAS"},"system":"urn:oid:2.16.840.1.113883.2.1.3.12.1.1","value":"2005380"},{"use":"usual","type":{"text":"INTERNAL"},"system":"urn:oid:1.2.840.114350.1.13.520.3.7.2.698084","value":"    Z11449"}],"active":true,"name":[{"extension":[{"valueCode":"NL4","url":"http://hl7.org/fhir/StructureDefinition/humanname-assembly-order"}],"use":"official","text":"BcnScTenAugTwentyOne UPGRADETEST","family":"UPGRADETEST","given":["BcnScTenAugTwentyOne"],"_family":{"extension":[{"valueString":"UpgradeTest","url":"http://hl7.org/fhir/StructureDefinition/humanname-own-name"}]}},{"extension":[{"valueCode":"NL4","url":"http://hl7.org/fhir/StructureDefinition/humanname-assembly-order"}],"use":"usual","text":"BcnScTenAugTwentyOne UPGRADETEST","family":"UPGRADETEST","given":["BcnScTenAugTwentyOne"],"_family":{"extension":[{"valueString":"UpgradeTest","url":"http://hl7.org/fhir/StructureDefinition/humanname-own-name"}]}}],"gender":"female","birthDate":"2019-09-21","deceasedBoolean":false,"maritalStatus":{"text":"Single"}}
{"resourceType":"Patient","id":"ezER-U3fAMP-WvI-Fc8V9wQ3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/CodeSystem/v3-NullFlavor","code":"UNK","display":"Unknown"},"url":"ombCategory"},{"valueString":"Unknown","url":"text"}],"url":"http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"},{"extension":[{"valueString":"Unknown","url":"text"}],"url":"http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"},{"valueCodeableConcept":{"coding":[{"system":"urn:oid:1.2.840.114350.1.13.520.3.7.10.698084.130.657370.334258","code":"male"}]},"url":"http://open.epic.com/FHIR/StructureDefinition/extension/legal-sex"},{"valueCodeableConcept":{"coding":[{"system":"urn:oid:1.2.840.114350.1.13.520.3.7.10.698084.130.657370.334258","code":"male"}]},"url":"http://open.epic.com/FHIR/StructureDefinition/extension/sex-for-clinical-use"},{"valueCode":"M","url":"http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex"}],"identifier":[{"use":"usual","type":{"text":"EPIC"},"system":"urn:oid:1.2.840.114350.1.13.520.3.7.5.737384.0","value":"E9411"},{"use":"usual","type":{"text":"EXTERNAL"},"system":"urn:oid:1.2.840.114350.1.13.520.3.7.2.698084","value":"Z11522"},{"use":"usual","type":{"text":"FHIR"},"system":"http://open.epic.com/FHIR/StructureDefinition/patient-dstu2-fhir-id","value":"T2oDrcOUw0w.1ZUdthu24c7V95lTOTEOUPRqBqGn99KEB"},{"use":"usual","type":{"text":"FHIR STU3"},"system":"http://open.epic.com/FHIR/StructureDefinition/patient-fhir-id","value":"ezER-U3fAMP-WvI-Fc8V9wQ3"},{"use":"usual","type":{"text":"PAS"},"system":"urn:oid:2.16.840.1.113883.2.1.3.12.1.1","value":"2005452"},{"use":"usual","type":{"text":"INTERNAL"},"system":"urn:oid:1.2.840.114350.1.13.520.3.7.2.698084","value":"    Z11522"}],"active":true,"name":[{"extension":[{"valueCode":"NL4","url":"http://hl7.org/fhir/StructureDefinition/humanname-assembly-order"}],"use":"official","text":"Fred TEST","family":"TEST","given":["Fred"],"_family":{"extension":[{"valueString":"Test","url":"http://hl7.org/fhir/StructureDefinition/humanname-own-name"}]}},{"extension":[{"valueCode":"NL4","url":"http://hl7.org/fhir/StructureDefinition/humanname-assembly-order"}],"use":"usual","text":"Fred TEST","family":"TEST","given":["Fred"],"_family":{"extension":[{"valueString":"Test","url":"http://hl7.org/fhir/StructureDefinition/humanname-own-name"}]}}],"telecom":[{"system":"phone","value":"07594 832748","use":"mobile"},{"system":"email","value":"[email protected]"}],"gender":"male","birthDate":"1967-01-19","deceasedBoolean":false,"address":[{"use":"old","line":["123 Anywhere"],"city":"Bristol","postalCode":"BS1 6JY","country":"ENG"},{"use":"home","line":["123 Anywhere"],"city":"Bristol","postalCode":"BS1 6JY","country":"ENG","period":{"start":"2022-01-19"}}],"maritalStatus":{"text":"Single"},"communication":[{"language":{"coding":[{"system":"urn:ietf:bcp:47","code":"en","display":"English"}],"text":"English"},"preferred":true}]}

I am opening ndjson file and trying to loop through the dictionaries to pick all the patient's id, birthDate and deceasedDateTime

Python Code to read ndjson file

import ndjson 

 with open('Patients.ndjson') as f:
   data = ndjson.load(f)
  return render_template('home.html', patient =  data)

Html Template This for loop displays all the items and values in html page.

How do i only pick specific values id, birthDate and deceasedDateTime from the given dictionaries?

{% for i in patient : %}
{% for item, value in i.items(): %}
     <tr>
       <td>{{ item }}</td>
         <td>{{ value }}</td>
      </tr>
     {% endfor %}
{% endfor %}

CodePudding user response:

See the section about variables for jinja2, you only need a small change:

main.py

from flask import Flask, render_template
import ndjson 


app = Flask(__name__)


template = "tmp.html"


@app.route('/')
def main():
    with open('patients.ndjson') as f:
        patients = ndjson.load(f)

    return render_template(template, patients=patients)



if __name__ == '__main__':
    app.run(debug=True)```

tmp.html

{% for patient in patients %}
    <tr>
        <td>{{ patient.id }}</td>
        <td>{{ patient.birthDate }}</td>
        <td>{{ patient.deceasedDateTime }}</td>
    </tr></br>
{% endfor %}

EDIT

As an FYI, when loading an ndjson in python, it becomes a list of dicts. In your template, you need to iterate over the list, and for each dict item get the correct value by using the key as shown in my example.

You could use dict.items() but this gives you all the key/value pairs unordered...

fe, using the following will print out the whole JSON for each line in the NDJSON:

{% for patient in patients %}
    {% for key,val in patient.items()  %}
        {{ key }}: {{val}}</br>
    {% endfor %}
{% endfor %}
  • Related