I try to scrape email to scrape email but it give me none any solution kindly recommend us these is page link https://www.avocats-lille.com//fr/annuaire/avocats-du-tableau-au-barreau-de-lille/2?view=entry

import requests
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from time import sleep

headers ={
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36'
}
base_url='https://www.avocats-lille.com/'
url = 'https://www.avocats-lille.com/fr/annuaire/avocats-du-tableau-au-barreau-de-lille?view=entries'
driver = webdriver.Chrome("C:\Program Files (x86)\chromedriver.exe")
driver.get(url)
soup = BeautifulSoup(driver.page_source, "html.parser")
tra = soup.find_all('h2',class_='title')
productlinks=[]
for links in tra:
    for link in links.find_all('a',href=True):
        comp=base_url link['href']
        productlinks.append(comp)
        
for link in productlinks:
    r =requests.get(link,headers=headers)
    soup=BeautifulSoup(r.content, 'html.parser')
    sleep(5)
    details=soup.find_all("div",class_="item col-5")
    for detail in details:
        email=soup.find('a[href^="mailto"]')
        print(email)

CodePudding user response：

Links you are looking for are not inside the tra (title) elements.
You should change the code as following to make it working:

tra = soup.find_all('div',class_='item')

CodePudding user response：

The Email address is within the following element:

<a href="mailto:[email protected]">[email protected]</a>

Solution

Using Selenium to print the Email address i.e. the innertext attribute you can use either of the following locator strategies:

• Using css_selector:

  driver.execute("get", {'url': 'https://www.avocats-lille.com//fr/annuaire/avocats-du-tableau-au-barreau-de-lille/2?view=entry'})
  print(driver.find_element("css selector", 'a[href^="mailto"]').text)
  

• Using xpath:

  driver.execute("get", {'url': 'https://www.avocats-lille.com//fr/annuaire/avocats-du-tableau-au-barreau-de-lille/2?view=entry'})
  print(driver.find_element("xpath", '//a[starts-with(@href, "mailto")]').text)
  

• Console Output:

  [email protected]