Selenium can't find all the tags in a webpage. It finds only one, but there are two, and I need those two to make my script work. This is the link. I tried using WebDriverWait, but it still returned me only one of the tags. I tried searching for different divs, but I kept returning me only one. I think it has to do with some of the classes that the second div has, and the first one doesn't, but I am really stuck.
full_image_link = driver.find_element_by_class_name('sc-1qpw8k9-3')
full_image_link.click()
full_image_link_ = WebDriverWait(driver, 10).until(
EC.presence_of_all_elements_located(
(By.CLASS_NAME, 'sc-1qpw8k9-3')
)
)
CodePudding user response:
You can use the api to retrieve the image links, no need to scrape:
import requests
r = requests.get('https://www.pixiv.net/ajax/illust/93333481/pages?lang=en').json()
image_links = [i['urls']['original'] for i in r['body']]
Result:
['https://i.pximg.net/img-original/img/2021/10/10/23/35/09/93333481_p0.jpg',
'https://i.pximg.net/img-original/img/2021/10/10/23/35/09/93333481_p1.jpg']