Data in to the csv file give nothing-CodePudding

WHEN I PRINT the data into data frame they will print all the data when I make the csv file they will give me nothing in csv file

import requests
from bs4 import BeautifulSoup
import pandas as pd

headers ={
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36'
}
r =requests.get('https://web.archive.org/web/20141113155415/http://www.alexa.com/topsites/category/Top/Recreation/Living_History/By_Historical_Region')
soup=BeautifulSoup(r.content, 'lxml')
main=soup.find_all('div',class_='desc-container')
web={}
for mains in main:

    link=mains.find('p',class_='desc-paragraph').text
    web['link']=link
    des=mains.find('div',class_='description').text
    web['description']=des
    
    
    df = pd.DataFrame([web])
    print(df)

CodePudding user response：

Try to put the dictionary with data into a list and each iteration create new dictionary. For example:

import requests
import pandas as pd
from bs4 import BeautifulSoup

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
}
r = requests.get(
    "https://web.archive.org/web/20141113155415/http://www.alexa.com/topsites/category/Top/Recreation/Living_History/By_Historical_Region"
)

soup = BeautifulSoup(r.content, "lxml")
main = soup.find_all("div", class_="desc-container")

all_data = []
for mains in main:
    web = {}
    link = mains.find("p", class_="desc-paragraph").text
    des = mains.find("div", class_="description").text

    web["link"] = link
    web["description"] = des
    all_data.append(web)

df = pd.DataFrame(all_data)
print(df.to_markdown())
df.to_csv("data.csv", index=False)

Prints:

	link	description
0	Texrenfest.com	Texas Renaissance Festival runs October and November at Plantersville, Texas.
1	Rennfest.com	A recreation of a 16th century English village named Revel Grove in Crownsville. Includes a sch... Moreedule, employment, craft and vendor applications, photographs, entertainment descriptions, and directions.
2	Plimoth.org	Provides information about Plimoth Plantation, the living history museum of the seventeenth cen... Moretury in Plymouth, Massachusetts. Also serves as a resource by which visitors may learn more about the Pilgrim Story, the history of Plymouth Colony (1620-1692), the Wampanoag Indians, Thanksgiving, and find links to related sites on the Web.
3	Royalfaires.com/carolina/	Annual event in Charlotte, North Carolina. Includes events schedule, ticket information, map, a... Moreccommodations, and contact information.
4	Renfestival.com	Located in Harveysburg, OH. Visit the 16th Century and see Jousting , Robin Hood, and Queen Eli... Morezabeth I.
5	Kingrichardsfaire.net	16th Century festival in Carver, Massachusetts. Typically runs August through mid October.
6	Rileysfarm.com	Riley's Farm is a working Southern California apple orchard and living history farm. Arrange a ... Morefield trip, or read about Colonial Christmas.
7	Bostonmassacre.net	Includes resource directory on the event, including pictures, documents, location, trial, parti... Morecipants, and timeline.
8	Renfair.com/NY/	Offers continuous performances by jugglers, jesters, minstrels and magicians, as well as jousts... More by knights in armor on charging horses located in Sterling Forest, near Tuxedo, NY.
9	Piratemerch.com	Specializes in pirate fashions, clothing, T-shirts, and gifts.
10	Renfaire.com	Provides background details of language, dress, customs and activities.
11	Okcastle.com	Experience the delights of Castleton; jousting, live chess matches, a masked ball, and the King... More's feast. Join King Henry VIII the first three weekends of May.
12	Livinghistory.co.uk	A resource for British re-enactors. Includes information on history, costumes and music, events... More listings, forums and free webspace for re-anactors.
13	Nps.gov/casa	Features information on accessibility, events, educational programs, management, news, history ... Moreand culture.
14	Asf.net	Attracting more than 300,000 visitors a year, ASF produces 14 world-class productions annually,... More including classical works, contemporary plays, lavish musicals, and new works commissioned by the Festival's Southern Writers' Project.
15	Kipar.org	An impressive site containing information on both the French and English Baroque periods.
16	Larp.com/legioxx/	Ancient Roman army and civilian life in the Britain in the First Century is reenacted for publi... Morec events and educational outreach. Based in Washington, DC; general and contact information.
17	Missionsanluis.org	Contains materials from the early settlement and capital of La Florida from 1656 to 1704. It in... Morecludes on-line collection of materials from the museum, and directions.
18	Ren-fest.com	Producer of the Florida Renaissance Festival and the Kyng's Company Renaissance Faire.
19	Golondrinas.org	The Ranch of the Swallows, a Spanish Colonial living history museum in Santa Fe, New Mexico. So... Moreldados (1820 or so) and Colonial civilians can be seen in the pictures.
20	Legionxxiv.org	Pennsylvania-based group re-enacting a Roman Legion. Equipment standards, schedule of events, p... Morehotos of re-enactments, background information on Rome.
21	Regia.org	Anglo-Saxon, Viking, Norman and British Living History 950-1066AD.
22	Pyracy.com	An online community enamoured with pirates. Includes a forum and some images.
23	Noquartergiven.net	Group portrays the pirate life for Faires, reenactments, museum meetings, and similar events; m... Moreembership information, events schedule, image gallery.
24	Warhorse.com	Group from the War Horse Farm in Sarasota, Florida, have jousted professionally at Faires and s... Moreimilar events since 1982, using Belgian, Percheron, Shire, and Clydesdale horses and full suits of replica Sixteenth Century tilting armor which allows riders to engage in an authentic full contact joust. Events schedule.

and saves data.csv (screenshot from LibreOffice):