web scraping returns an empty list-CodePudding

 import requests
 from bs4 import BeautifulSoup

 result = requests.get('https://www.indeed.com/?vjk=5bc59746be36d8d0')
 source = result.content
 soup = BeautifulSoup(source, "lxml")

 job_titles = soup.find_all("a", {"class": "jcs-JobTitle"})

 print(job_titles)

The problem here that printing job_titles returns an empty list instead of the job titles in the web site

please help me fix this problem and any help would be appreciated

CodePudding user response：

When I first went to the URL you're requesting, I was shown a search page with no jobs listed. It was only after I submitted a search that the page was populated with results. When I returned to the original URL again, the page was still populated (possibly with cached results). The blank page is probably what you're getting back when you get the page from requests.

Try using the full URL with parameters that the browser forwards you to after a search. For example, the URL https://www.indeed.com/jobs?q=data engineer&l=Raleigh, NC&vjk=b971ec43674ab50e gives me back 15 job title links.