BeautifulSoup: get x and y values from a plotly plot?-CodePudding

How would do you get the values from the red line in this plot below?

Second graph:

my attempt:

import pandas as pd
import numpy as np
import requests
from bs4 import BeautifulSoup

header = {
  "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36",
  "X-Requested-With": "XMLHttpRequest"
}

link = 'https://dash.gallery/python-docs-dash-snippets/_dash-update-component'

r = requests.get(link, headers=header, timeout=6.0)
soup = BeautifulSoup(r.text, "html5lib")
soup

I understand my above attempt is just obtaining the webpage, but the elements that contain the data are loaded separately (shown in the screenshot). I can see the data to the red line is in the response section (circled in the screenshot).

My end result would ideally be a pandas dataframe with the x and y coords of the red line.

CodePudding user response：

to obtain the curl request, right click copy cURL.

place this into the python code below to obtain a dataframe of the plotly data

from pandas import json_normalize
import json
var = !curl 'https://dash.gallery/python-docs-dash-snippets/_dash-update-component' \
  -H 'authority: dash.gallery' \
  -H 'sec-ch-ua: "Google Chrome";v="95", "Chromium";v="95", ";Not A Brand";v="99"' \
  -H 'accept: application/json' \
  -H 'x-csrftoken: undefined' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'content-type: application/json' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36' \
  -H 'sec-ch-ua-platform: "Windows"' \
  -H 'origin: https://dash.gallery' \
  -H 'sec-fetch-site: same-origin' \
  -H 'sec-fetch-mode: cors' \
  -H 'sec-fetch-dest: empty' \
  -H 'referer: https://dash.gallery/python-docs-dash-snippets/line-charts' \
  -H 'accept-language: en-US,en;q=0.9' \
  --data-raw '{"output":"line-charts-line-chart.figure","outputs":{"id":"line-charts-line-chart","property":"figure"},"inputs":[{"id":"line-charts-checklist","property":"value","value":["Americas","Oceania"]}],"changedPropIds":[]}' \
  --compressed
unloaded = json.loads(var[0]) 
json_normalize(unloaded['response']['line-charts-line-chart']['figure']['data'])