This is my Code `

from attr import attr
import requests
from bs4 import BeautifulSoup
import csv

datas = []
key = 'sepatu'
jenis = 'teplek'
url = 'https://website.com/search/?term={} {}'.format(key,jenis)
headers = {
    'user-agent' : 'Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0'
}
req = requests.get(url, headers=headers)
soup = BeautifulSoup (req.text, 'html.parser')
sepatu = soup.find_all('div', 'element_1')
for it in sepatu:
    harga = it.find('div','element').__str__
    datas.append([harga])
hasil = ['Harga'] 
write = csv.writer(open('result/{}_{}.csv'.format(key,jenis), 'w', newline=''))
write.writerow (hasil)
for d in datas: write.writerow(d)

this is Output from this code

Column A
<bound method Tag.unicode of Rp 88.000 >
<bound method Tag.unicode of Rp 200.000 >

how to turn that output to this output

Column A
Rp 88.000
Rp 200.000

i was try harga = it.find('div','element').__str__ to harga = it.find('div','element').text but i got error AttributeError: 'NoneType' object has no attribute 'text'

i try to learn web scraping python selenium but i got block by output into text and i expecting i want all the output into text

• python
• selenium
• web
• web-scraping

Share Follow asked 8 hours ago RezkyRezky 3855 bronze badges

CodePudding user response：

You can add .text in this line

harga = soup.find("div", {"class": "db gM ei b hE be f16-360-o ff vb uT ellipsis-1"}).text

Then you will get an output like this

Nama Sepatu	Harga
Sepatu A	Rp.24.000

CodePudding user response：

Looks like the problem is with the commands you use to locate the elements. If element is not found, BS returns None for them.
What mean element_1 and element in this lines?:

sepatu = soup.find_all('div', 'element_1')

and

harga = it.find('div','element').__str__

If these are IDs, you need to set it this way:

sepatu = soup.find_all('div', {'id': 'element_1'})

If it's a class name:

sepatu = soup.find_all('div', {'class': 'element_1'})

Or any other attribute:

sepatu = soup.find_all('div', {'attr_name': 'element_1'})

When an element is found you can use .text property to get its text