My aim is parse "funpay.com"'s offer page. It has to be easy, cause all offer names are inside the same class 'tc-item'
.
However I can't use bs4 requests, because this page loads only if you're logged in, which I'm doing via cookies (selenium pickle
).
Idk how to make it at all, so I'll appreciate any hints.
The code I tried:
driver.get("https://funpay.com/orders/trade")
soup = bs(driver.page_source, 'html.parser')
try:
paid = soup.find_all('a', class_='tc-item')
for sold in paid:
title = sold.find('div', class_='tc-order') # inside 'a',
# prints the code of offer
print(title)
except Exception as ex:
print(ex)
CodePudding user response:
Based on the rather thin starting point, I suspect it's an error that occurs during iterations, so here's what I would do.
In order not to discard everything directly, check inside the loop whether the element you are looking for is available or not and let output the result accordingly.
...
soup = bs(driver.page_source, 'html.parser')
paid = soup.find_all('a', class_='tc-item')
for sold in paid:
title = sold.find('div', class_='tc-order').text if sold.find('div', class_='tc-order') else 'no element found'
print(title)