Scrapy framework crawl data, crawl, and Settings to add, let crawl data stored in the local CSV format, each run only generates a header, not continue to write other data, and if changed to excel mode, can the normal written to the file,



# Settings set as follows:

BOT_NAME='dangdang'

SPIDER_MODULES=[' dangdang. Spiders']
NEWSPIDER_MODULE='dangdang. Spiders'


# Crawl responsibly by identifying yourself (and your website) on the the user-agent
USER_AGENT='Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36'

# Obey, robots. TXT rules
ROBOTSTXT_OBEY=False
FEED_URI='work dangdang. CSV'
FEED_FORMAT='CSV'
FEED_EXPORT_ENCODING='ANSI'

DOWNLOAD_DELAY=0.5

ITEM_PIPELINES={
'dangdang. Pipelines. DangdangPipeline: 300,
}

CodePudding user response: