Coding (python) newbie here.I am trying to make a small project where I gather news headlines from an API.The API I am trying to use is newsapi.org. Every time I enquire the API, It gives me a a long list of news articles,which I add to a database . But when I enquire again after some time gap , I get a new article(S) plus many, if not all the previous articles. How do I avoid adding duplicates to the database? I thought of iterating through the last few new entries and see if such articles already exist but somehow I feel that is not the way.Any help is appreciated. Thank you

CodePudding user response：

The API you want to use includes a from parameter that you can set to filter the articles returned and only get the ones that are more recent.

If you store the current time each time you gather data from the api, you can use it in the from parameter on the next api call you will do to only get what has been published in between the two calls.