Home > database >  Problem with naming the yield return in python
Problem with naming the yield return in python

Time:11-01

I am trying to scrape a website using python and scrapy but I have issues with saving the result.

error log i receive:

    yield result = {
                 ^
SyntaxError: invalid syntax

When i remove the "result = ", I don't get any error but the reason I am doing that is to save the result as a variable which I use at the last part of the code in "f.write(result)"

The code goes below:

import scrapy

class ExampleSpider(scrapy.Spider):
    name = "ufcspider"

    start_urls = [
        'http://quotes.toscrape.com/page/1/',
    ]

    def parse(self, response):
        for quote in response.css('div.quote'):
            yield result = {
                'text': quote.css('span.text::text').get(),
                'author': quote.css('small.author::text').get(),
                'link': 'http://quotes.toscrape.com'   quote.css("span a::attr(href)").get(),
                'tags': quote.css('div.tags a.tag::text').getall(),
            }

        next_page = response.css("li.next a::attr(href)").get()
        if next_page is not None:
            next_page = response.urljoin(next_page)
            yield scrapy.Request(next_page, callable=self.parse)

        page = response.url.split("/")[-2]
        filename = f'quotes-{page}.json'
        with open(filename, 'wb') as f:
            f.write(result)
        self.log(f'Saved file {filename}')

CodePudding user response:

First define result, next yield it

result = { ... }

yield result
  • Related