After web scraping using selenium, I got weird results in my csv file.. Instead of having specific c-CodePudding

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By

import time
import csv


driver = webdriver.Chrome('/Users/myname/Desktop/web_crawling/chromedriver')

driver.get('https://www.naver.com')
time.sleep(2)


driver.find_element(by=By.CSS_SELECTOR, value='a.nav.shop').click()

search = driver.find_element(by=By.CSS_SELECTOR,value='._searchInput_search_input_QXUFf')
search.click()

search.send_keys("아이폰 13")
search.send_keys(Keys.ENTER)

before_h = driver.execute_script("return window.scrollY")

while True:
    driver.find_element(by=By.CSS_SELECTOR, value='body').send_keys(Keys.END)
    time.sleep(1)

    after_h = driver.execute_script("return window.scrollY")

    if after_h == before_h:
        break
    before_h = after_h

#create csv file
f = open(r"/Users/yungijeong/Desktop/web_crawling/data.csv", 'w', encoding='UTF8')
csvWriter = csv.writer(f)

items = driver.find_elements(by=By.CSS_SELECTOR, value=".basicList_info_area__17Xyo")

for item in items:
    names = item.find_elements(by=By.CSS_SELECTOR,  value=".basicList_link__1MaTN")
    for name in names:
        print(name.text)

    try:
        prices = item.find_elements(by=By.CSS_SELECTOR, value=".price_num__2WUXn")
        for price in prices:
           print(price.text)
    except:
        print("판매중단")
    links = item.find_elements(by=By.CSS_SELECTOR, value=".basicList_title__3P9Q7 > a")
    for link in links:
        print(link.get_attribute('href'))
    print(name, price, link)

    #adding inside the csv files

    csvWriter.writerow([name, price, link])

f.close()

Here, I am trying to web scrape the details and prices of iphones in a Koran shopping website. I made my code so that the webdriver automatically goes into the site and get all the details (such as prices and link for the products). And in the end, it's supposed to make a csv file and paste all the data that is scraped inside there.

The code works perfectly, but when I export it to a csv file, it looks like this: The result in csv

The contents seem like they weren't exported properly.. Every single code looks like a HTML code...Did any of you have the same issue? in the terminal, it looks like the webdrvier correctly distinguished the data, but as a result, it exports it in a weird way. Please share if you had the same issue!!

CodePudding user response：

I think all problem is that you print() values but you don't assign to variables.

You have

print(name.text)
print(price.text)
print(link.get_attribute('href'))

but you forgot

name = name.text
price = price.text
link = link.get_attribute('href')

OR you should write

csvWriter.writerow([name.text, link.text, link.get_attribute('href')])