Empty CSV when writing selenium data-CodePudding

I'm making my first experience with Selenium and doing a few tutorials on a well-known video platform. It works quite reliable most of the time. However, I had a problem with a few pages that the CSV is created but no export of the data is made. The CSV is "touched", but it does not export the data that is displayed in a normal print.

Can anyone help me where the problem is with this script?

#_*_coding: utf-8_*_


from selenium import webdriver
import selenium
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
import csv
import os

os.chdir("C:\Selenium")
PATH = "chromedriver.exe"
driver = webdriver.Chrome(PATH)

driver.get("https://twitter-trends.iamrohit.in/")

try: 
    main = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.CLASS_NAME, "panel-body"))
    )
 
    main = (main.text)
 
    f = open('twitter.csv', 'wb')
    print(main, file = f)
    f.close()
    
    #print(main)

except:
    driver.quit()

driver.quit()

Python Version 3.7.4, Selenium Versionm 3.141.0, Windows 10

CodePudding user response：

Debugging your code you are getting the data correctly.

   main = (main.text)
   print(main)
   f = open('twitter.csv', 'wb')

So the error is when you are writing to the output file. Replacing your code for

main = (main.text)

with open('twitter.txt', 'wb', encoding='utf-8') as file1:
# Writing data to a file
    file1.writelines(main)

will work, if you check the print you have Chinese characters that will make fail the writing in the output file.

CodePudding user response：

To scrape the

References

You can find a couple of relevant detailed discussions in: