Python: reading CSV file line by line, how to detect end?-CodePudding

I read large CSV file (millions of records) by this script. How do I detect the file is at end?

import csv
f = open("file.csv", newline='')
csv_reader = csv.reader(f)

while true:
    do something with next(csv_reader)[6]

CodePudding user response：

The obvious solution is to loop over csv_reader, as suggested by this answer. If that is not practical, the docs for the next function say:

Retrieve the next item from the iterator by calling its __next__() method. If default is given, it is returned if the iterator is exhausted, otherwise StopIteration is raised.

thus giving you two ways of detecting the end.

CodePudding user response：

The csv.reader will read the file entirely and store it in the variable which is also an iterable. For reading "line by line", you need this:

for row in csv_reader:
    do something

If you directly want the last line:

with open(‘file_name.csv’,’r’) as file: 
    data = file.readlines() 
lastRow = data[-1]

This will be quite slow and memory consuming. Alternative is using pandas.