Home > Enterprise >  Cannot Scrap li and put result in dict in BS
Cannot Scrap li and put result in dict in BS

Time:11-04

I'm trying to scrape li elements and add results to a dictionary so that my code always get what is inside li and finally I'll be creating df from this dictionary.

my code :

link = 'https://www.propertyfinder.eg/en/plp/rent/apartment-for-rent-cairo-hay-el-maadi-degla-street-207-3455087.html'
r = requests.get(link , headers=headers)
soup = bs(r.content,'lxml')
data = {}
data1 = soup.find('ul')
s = [li.get_text(strip=True) for li in data1.select("li")]
data.update(dict(s))
data

I got this error :

ValueError: dictionary update sequence element #0 has length 23; 2 is required

To make it clear i want to have a dictionary like this :

{'Property type' : 'Apartment',
 'Property size' : '2,368 sqft  /  220 sqm',
 'Bedrooms' : '3',
 'Bathrooms':4
}

CodePudding user response:

You need to get the specific elements that contain the keys and values you want to put into the dictionary. The keys are in the <span> elements, and the values are in <div >.

You can use a dictionary comprehension to create the dictionary directly, instead of using a list comprehension followed by dict() and dict.update().

data = {li.find('span').get_text(strip=True): li.find('div', class_="property-facts__value").get_text(stripe=True) 
        for li in data1.select("li")}
  • Related