How to repeat "for" loop based on response?-CodePudding

Currently, I have a program that reads data from a text file, parses it with a function,and will output it to another file. My issue is that my function uses get requests, and gets rate-limited frequently, returning a 429. I'm trying to make the program detect the 429 response, wait 300 seconds, then repeat from the line that previously failed. I've tried a nested while loop inside the for loops but I can't seem to get it working. My code is shown below:

for line in lines: # lines is from input file readlines()
    data = item_data(line)  #item_data returns (data, response code)
    if data[1] != 429:
        with open('out.csv', 'a') as f:
            f.write("Parsed data here!")
        print('Success!')
    else:
        time.sleep(300)
        print(Failed)
        #repeat item_data from line
f.close()

An example output of what I want it to do would be:

1 
Success!
2
Success!
3
Success!
4
Failed
4
Success!
5
Success!

CodePudding user response：

You want to retry the same line again.

for line in lines: # lines is from input file readlines()
    while True:
        data = item_data(line)  #item_data returns (data, response code)
        if data[1] != 429:
            break
        # else:
        print(Failed)
        #do something
        time.sleep(300)
    with open('out.csv', 'a') as f:
        f.write("Parsed data here!")
    print('Success!')

CodePudding user response：

Try this (no need to close a file if you're using a with statement):

with open('some_file.txt') as x: # Replace 'some_file.txt' with your file
    lines = x.readlines()
    for i, line in enumerate(lines):
        print(i   1) # Will print 1, 2, 3, etc.
        data, code = item_data(line)
        if code != 429:
            with open('out.csv', 'a') as f:
                f.write(str(data)) # Or whatever you want to write
            print('Success!')
        else:
            print(f'Failed with code {code}. Waiting 300s')
            time.sleep(300)

CodePudding user response：

Your for command clause must be:

data = item_data(line)
while data[1]==429:
    print('fail')
    time.sleep(300)
    data = item_data(line)
with open('out.csv', 'a') as f:
    f.write("Parsed data here!")
print('Success!')

It is the easiest way that you can apply just change your condition