JSON request returns a blank response randomly causing script to fail-CodePudding

I have a python script that is failing when my request returns a blank JSON response. The code loops through the function repeatedly and is successful 99% of the time, but fails every once in a while when a blank JSON response is received.

i = 0
while i < 1000:
r = 0
while r < 4:
    time.sleep(5)
    response = c.get_quote(symbol)
    jsonData = response.json()
    for key in jsonData:        
        jsonkey = key
    if jsonkey == symbol:
        print (i)
        print("you are good", response)
        print(jsonData)
        print ()
        break
    else:
        print("you have a problem, jsonkey=")
        print()
        print (jsonData)
        print()
    r =  1
current_price = response.json()[symbol]["lastPrice"]
i  = 1

The 'While r < 4:' loop was added in an attempt to add error handling. If I can figure out what to trap on, I would retry the "response = c.get_quote(symbol)" but the blank JSON response is slipping past the 'if jsonkey = symbol' logic.

The error message received is "current_price = response.json()[symbol]["lastPrice"] KeyError: 'NVCR'"

and the output from print (jsonData) is: {} as opposed to a healthy response which contains the symbol as a key with additional data to follow. The request is returning a response [200] so unfortunately it isn't that simple...

CodePudding user response：

Instead of validating the key with jsonkey == symbol, use a try-except block to catch the blank response errors and handle them.

For instance:

i = 0
while i < 1000:

time.sleep(5)
response = c.get_quote(symbol)
jsonData = response.json()
try:
    for key in jsonData:        
        jsonkey = key
    if jsonkey == symbol:
        print (i)
        print("you are good", response)
        print(jsonData   "\n")
        break
except:
    print("you have a problem \n")
    print (jsonData   "\n")

current_price = response.json()[symbol]["lastPrice"]
i  = 1

@DeepSpace is also likely correct in the comments. My guess is that the server that you're pulling json data from (nsetools?) is throttling your requests, so it might be worth looking deeper in their docs to see if you can find a limit, and then use time.sleep() to stay under it.

Edit: If you are using nsetools, their api seems to be built by reverse-engineering the api that the nse website is built on and performing json api calls to urls such as this one (these can be found in this source code file). Because of this, it's not documented what the rate limit is, as this data is scraped directly from NSE and subject to their rate limit. Using this data is against NSE's terms of use (unless they have express written consent from the government of India which for all I know nsetools has, but I assume you do not.)

CodePudding user response：

OK so thanks to @DeepSpace and @CarlHR I think I have a solution but it still seems like there is too much code for what I am trying to accomplish. This works:

i = 0
while i < 1000:
r = 1
while r < 5:
    time.sleep(1)
    response = c.get_quote(symbol)
    jsonData = response.json()
    try:
        current_price = response.json()[symbol]["lastPrice"]
        print ("Looks Good, moving on")
        break
    except KeyError:
        print ("There was an problem with the JSON response, 
        trying again.  retry number:", r)
        print (jsonData)
        print ()
        r  = 1
     
i  = 1
print ("Moving on to the next iteration")