Home > database >  Python Scrapy unable to get each class name with its value from each div
Python Scrapy unable to get each class name with its value from each div

Time:04-25

I'm trying to get each class names with its corresponding value from each div listing.

From this page: https://www.dotproperty.id/en/properties-for-sale/riau-islands/batam?page=1

Here's the html for the class names and values:

<ul >
<li><i ></i> <span>3</span></li>
<li><i ></i> <span>2</span></li>
<li><i ></i> <span>100 m<sup>2</sup></span></li>
</ul>

So far the closest I got is that it prints each classes with its value side by side using getall(). But, it gives me the whole page.

for e in response.css('div.description-block'):
     for item in e.xpath('.//ul[@]'):
         print(item.xpath('.//li/i/@class | .//li/span/text()').getall())

Output:

['icon icon-bedroom', '3', 'icon icon-dp-icon-24', '2', 'icon icon-dp-icon-26', '100 m']
['icon icon-bedroom', '2', 'icon icon-dp-icon-24', '1']
['icon icon-bedroom', '6', 'icon icon-dp-icon-24', '5', 'icon icon-dp-icon-26', '198 m']
['icon icon-bedroom', '4', 'icon icon-dp-icon-24', '5', 'icon icon-dp-icon-26', '450 m']
['icon icon-dp-icon-26', '336 m', ' ']

Trying get() gives each first class name of each div:

icon icon-bedroom
icon icon-bedroom
icon icon-bedroom
icon icon-bedroom
icon icon-dp-icon-26
(and so on ...)

I want the output to look like that for each loop.

['icon icon-bedroom', '3', 'icon icon-dp-icon-24', '2', 'icon icon-dp-icon-26', '100 m']

CodePudding user response:

Ok, so i solved my issue. Basically, i just needed to iterate inside the icon container and discriminate each icon by its names.

Looping inside the looping loop. :)

for a in accommodation:
        class_name = a.xpath('.//i/@class').getall()
        value = a.xpath('li/span/text()').getall()
        for name,val in zip(class_name,value):
            if 'bedroom' in name:
                bedrooms = val
            else:
                pass
            if 'icon-24' in name:
                bathrooms = val
            else:
                pass
            if 'icon-26' in name:
                land = val
            else:
                pass
  • Related