ScraperException: 4 requests to https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&send_error_codes=true&simple_quoted_tweets=true&q=(from:ZeeNewsEnglish) until:2023-01-12 since:2023-01-08 -filter:replies&count=100&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&ext=mediaStats,highlightedLabel failed, giving up.

I tried following code :

import snscrape.modules.twitter as sntwitter
import time

query5 = "(from:BBC) until:2023-01-12 since:2023-01-08 -filter:replies"

news = [query5]
tweets = []

for news_data in news:
    limit = 500
    for tweet in sntwitter.TwitterSearchScraper(news_data).get_items():

        # print(vars(tweet))
        # break
        if len(tweets) == limit:
            break
        else:
            tweets.append([tweet.date, tweet.username, tweet.content])
        
    time.sleep(2)

CodePudding user response：

Faced the same issue. I guess snscraper, made use of Twitter API (elevated access) in the backend. Twitter shut down all the bots which were fetching the API data. Twitter essentially wants you to make authentic use of their data. I suggest signing up on twitter's developer account and requesting the elevated environment. Notice the first line in error makes a call to Twitter API.

CodePudding user response：

you have to install the last version of snscrape 0.5.0.20230113.