Website doesnt return table when querying by scrapy-CodePudding

I am trying to query https://www.bankofindia.co.in/Home/BranchLocator?page=1 and when I open the website, it shows the table of data. And moving forward and backward in pages also shows and updates the table. But when querying via scrapy the table is not returned.

    start_urls = [
    "https://www.bankofindia.co.in/Home/BranchLocator?page={}".format(i)
    for i in range(1, 10)
]

def parse(self, response, **kwargs):
    """Parse response."""
    all_names = response.xpath(
        "//span[@id='BranchName']/text()",
    ).extract()
    for name in all_names:
        print(name)

    all_addresses = response.xpath(
        "//span[@id='Address']/text()",
    ).extract()

Am I doing something wrong or the website is not behaving correctly?

CodePudding user response：

Nothing is wrong here. Actually to get table data you must need to fill in the form and that filled in form will generate tables and that table data you have to scrape otherwise it's not possible. The table data with page numbers you are seeing that is dynamic even not generating from xhr api request.

PS: You need to fill in the following: Brunch Name, IFSC Code, State, City