Beautiful Soup parsing values from JSON-CodePudding

I'm working on a little project, I have this website: https://earthquake.usgs.gov/ws/designmaps/nehrp-2020.json?latitude=20.85&longitude=-156.5&riskCategory=IV&siteClass=Default&title=

And I need to parse the value of "sds" and "sd1".

I've tried doing with this code:

from urllib import request
from bs4 import BeautifulSoup
import json

url = "https://earthquake.usgs.gov/ws/designmaps/nehrp-2020.json?latitude=20.85&longitude=-156.5&riskCategory=IV&siteClass=Default&title="
html = request.urlopen(url).read()
soup = BeautifulSoup(html, "html.parser")

site_json = json.loads(soup.text)
print([d.get("sd1") for d in site_json["response"] if d.get("sd1")])

and in response I get:

Traceback (most recent call last):
  File "C:\Users\Bigboi\PycharmProjects\importpaska\main.py", line 10, in <module>
    print([d.get("sd1") for d in site_json["response"] if d.get("sd1")])
  File "C:\Users\Bigboi\PycharmProjects\importpaska\main.py", line 10, in <listcomp>
    print([d.get("sd1") for d in site_json["response"] if d.get("sd1")])
AttributeError: 'str' object has no attribute 'get'

Process finished with exit code 1

Can anyone help me?

CodePudding user response：

You can get the desired data using requests module only

    import requests
    import json

    url = "https://earthquake.usgs.gov/ws/designmaps/nehrp-2020.json?latitude=20.85&longitude=-156.5&riskCategory=IV&siteClass=Default&title="
    data=requests.get(url).json()
    
    # data=json.dumps(data)
    # with open('data.json','w') as f:
    #     f.write(data)
    
    sds=data['response']['data']['sds']
    sd1=data['response']['data']['sd1']
    print(sds)
    print(sd1)

Output:

0.59
0.38

CodePudding user response：

site_json["response"] is dictionary and you can not iterate that way and there's no key named sd1 in site_json["response"]. if you want to parse value of sd1 and sds, try this:

print(site_json["response"]["data"]["sd1"])

print(site_json["response"]["data"]["sds"])