I am trying to scrape data from this traffic camera map:

https://traffic.houstontranstar.org/layers/layers_ve.aspx?&inc=true&rc=true&lc=false&cam=true&dms=false&spd=true&tr=false&wthr=false&rfw=true&rm=false&nml=false&nmd=false

This is what it looks like: 1

The map shows the positions of a bunch of traffic cameras with these camera icons: 2

I am currently able to manually inspect element at each camera location and get the source link for the image in the camera's video feed. For example, I can get this link here for an example camera: https://www.houstontranstar.org/snapshots/cctv/913.jpg?arg=1661886693594

However, I also need to get the location of these cameras. Does anyone know if there is a way for me to get some sense of geographic location of these cameras on the map?

The second question I had was that if anyone knew of a good way for me to iterate upon the cameras? I'm completely new to web scraping so am currently just inspecting the elements of the website manually and going from there.

Any help would be great thanks!

CodePudding user response：

The following code is one way of obtaining those cameras locations (lat/lon):

import requests import pandas as pd

url = 'https://traffic.houstontranstar.org/data/layers/cctvSnapshots_json.js?arg=1661888238910'
r = requests.get(url)
df = pd.DataFrame(r.json()['cameras'])
print(df)

This returns a dataframe with 1126 rows × 8 columns:

[...]