I'm relatively new to python and have only done projects doing dataframe analysis. I am trying to learn web scraping to complete a personal proejct.
I'm practicing this basics and this is my current code:
import pandas as pd
from bs4 import BeautifulSoup
from selenium import webdriver
import requests
html_text = requests.get('https://www.pff.com/news/nfl-quarterback-rankings-all-32-starters-ahead-of-the-2021-nfl-season').text
soup = BeautifulSoup(html_text,'lxml')
players = soup.find('div',class_ = 'm-longform-copy')
for player in players:
name = players.h3.a.text
print(name)
When I run this, it just prints "Patrick Mahomes" repeatedly instead of going onto the next entry. I looked up a few other similar questions like this on here, but don't know the syntax well enough to apply it to my issue. Any help would be great!
CodePudding user response:
import pandas as pd
from bs4 import BeautifulSoup
from selenium import webdriver
import requests
html_text = requests.get('https://www.pff.com/news/nfl-quarterback-rankings-all-32-starters-ahead-of-the-2021-nfl-season').text
soup = BeautifulSoup(html_text,'lxml')
players = soup.select('h3 ')
for player in players:
name = player.text
print(name)
Output:
1. Patrick Mahomes, Kansas City Chiefs
2. Tom Brady, Tampa Bay Buccaneers
3. Aaron Rodgers, Green Bay Packers
4. Russell Wilson, Seattle Seahawks
5. Deshaun Watson, Houston Texans
6. Josh Allen, Buffalo Bills
7. Dak Prescott, Dallas Cowboys
8. Lamar Jackson, Baltimore Ravens
9. Matt Ryan, Atlanta Falcons
10. Baker Mayfield, Cleveland Browns
11. Matthew Stafford, Los Angeles Rams
12. Ryan Tannehill, Tennessee Titans
13. Derek Carr, Las Vegas Raiders
14. Kirk Cousins, Minnesota Vikings
15. Justin Herbert, Los Angeles Chargers
16. Ben Roethlisberger, Pittsburgh Steelers
17. Kyler Murray, Arizona Cardinals
18. Joe Burrow, Cincinnati Bengals
19. Ryan Fitzpatrick, Washington Football Team
20. Daniel Jones, New York Giants
21. Trevor Lawrence, Jacksonville Jaguars
22. Jimmy Garoppolo, San Francisco 49ers
23. Carson Wentz, Indianapolis Colts
24. Jameis Winston/Taysom Hill, New Orleans Saints
25. Justin Fields, Chicago Bears
26. Jared Goff, Detroit Lions
27. Cam Newton, New England Patriots
28. Sam Darnold, Carolina Panthers
29. Tua Tagovailoa, Miami Dolphins
30. Zach Wilson, New York Jets
31. Jalen Hurts, Philadelphia Eagles
32. Drew Lock, Denver Broncos
NFL Featured Tools