Nasa API pass the data so slow-CodePudding

I'm trying to get near-earth asteroid data from NASA API. And I'm getting the data I need but it's coming very slow. How can I optimize my code to get data quickly?

    @api_view(['GET'])
    def getDates(request, start_date, end_date):
        dates = []
        all_data = list()
        api_key = 'hpdCBykg6Bcho1SitxAcgAaplIHD1E0SzNLfvTWw'

        url_neo_feed = "https://api.nasa.gov/neo/rest/v1/feed?"
        params = {
            'api_key': api_key,
            'start_date': start_date,
            'end_date': end_date
        }
        response = requests.get(url_neo_feed, params=params)
        json_data = orjson.loads(response.text)
        date_asteroids = json_data['near_earth_objects']
        for date in date_asteroids:
            dates.append(date)
   
        # Splitting the data to make it more meaningful
        for date in dates:
            collection = json_data.get('near_earth_objects')
            all_dates = collection.get('{}'.format(date))
            all_data.append(all_dates)
        return Response(all_data)

CodePudding user response：

I would separate & time the fetch and parsing logic to determine first if it's the API that is slow or the parsing of the data.

CodePudding user response：

NASA's API apparently takes 8 seconds to respond for a 4-day period of data.

Your processing code takes a very short time (less than 0.01 seconds) to process that data.

There's not much you can do about NASA's API being slow, but you could cache the data for a given period locally, if that's okay for your application; subsequent requests for that range would then be near-instant.

You can use e.g. Django's cache (make sure it's configured to be something else than the DummyCache to see an effect):

from django.core.cache import cache

@api_view(['GET'])
def getDates(request, start_date, end_date):
    cache_key = f'nasa_neo_{start_date}_{end_date}'
    json_data = cache.get(cache_key)
    if not json_data:
        response = requests.get("https://api.nasa.gov/neo/rest/v1/feed?", params={
            'api_key': api_key,
            'start_date': start_date,
            'end_date': end_date
        })
        response.raise_for_status()
        json_data = orjson.loads(response.text)
        cache.set(cache_key, json_data, timeout=86400)

    date_asteroids = ...