I am new to web scraping. I want to scrape the data (comments and respective dates) from this web page It has pagination for pages.... This is the way I am doing

This code is working only on first page its give all comments and dates accordingly..but not from page 2.3.4..... which appears as we scroll to the buttom

How can I do that ...Thank you

CodePudding user response：

In your special case, each page has their own link. It is your base link and the page number with an hyphen (-) in between.

You can see this behaviour when clicking on the second page, compare your base-link with the link you have now: https://forum.donanimhaber.com/turkcell-25-000-dk-12-000-sms-70-gb-internet-12-ay-320-tl-ana-konu--151777938-2

(notice the -2 at the end)

One way to do it, would be to change your url in a for-loop, iterating up to 24 and scrape all of those pages individually.