Beautiful Soup error: 'NoneType' object has no attribute 'find

I am getting this error:

---> 14 content = s.find_all('p')
AttributeError: 'NoneType' object has no attribute 'find_all'

While running the script below:

r = requests.get('https://www.marketsandmarkets.com/Market-Reports/rocket-missile-market-203298804.html/')
soup = BeautifulSoup(r.content, 'html.parser')
s = soup.find('div', class_='entry-content') 
content = s.find_all('p')
print(content)

It's working for some URLs, but for other URLs it gives an attribute error. Not sure why this is happening.

CodePudding user response：

When soup.find does not find anything, it returns None. Guard your s.find_all call with if s: or if s in not None:.

So your code becomes:

r = requests.get('https://www.marketsandmarkets.com/Market-Reports/rocket-missile-market-203298804.html/')
soup = BeautifulSoup(r.content, 'html.parser')
s = soup.find('div', class_='entry-content') 
if s is not None:
    content = s.find_all('p')
    print(content) 
else:
    print("Didn't find what I was looking for...")

or maybe:

content = s.find_all('p') if s is not None else []
print(content)

Ref: https://crummy.com/software/BeautifulSoup/bs4/doc/#find