Output enumerate function to csv in Python-CodePudding

I am trying to understand how to enumerate through a csv file and output the results to another csv file. For example, I have a csv file that has 2 columns, group_id and node_id. I want to use this csv file, loop through it and output the group_id, node_id and a formatted string that mentions the node_id. My code does exactly this when I use the print function however when I attempt to write to another csv file, only the last row is written.

Here is my code:

import csv

with open('input.csv', 'r') as f:

    config_csv = csv.reader(f)

    for row, column in enumerate(config_csv):

        if row == 0:
            continue

        group_id = int(column[0])
        sub_id = column[1]
        node = f"The sub ID for this node is {sub_id}."
        full_output=[group_id, sub_id, node]
        print(full_output)

        with open('output.csv', 'w') as file:
            writer=csv.writer(file)
            writer.writerow(full_output)

csv file (input.csv):

GROUP_ID,SUB_ID
675233,111
877531,222
455632,333

And my print output is:

[675233, 111, 'The sub ID for this node is 111.']
[877531, 222, 'The sub ID for this node is 222.']
[455632, 333, 'The sub ID for this node is 333.']

However my output file (output.csv) only shows the last line:

[455632, 333, 'The sub ID for this node is 333.']

What am I doing wrong? Why doesn't it output the same to the csv file that I see in the print function?

CodePudding user response：

Open both files and write the lines as they are processed. It's not clear from your incorrect indentation but you are probably writing a new file each time through the loop so only end up with the last line.

Also, make sure to use newline='' when opening as per csv documentation:

import csv

# Need Python 3.10 for parenthesized context manager support.
# Use the following one-liner on older versions.
#with open('input.csv', 'r', newline='') as fin, open('output.csv', 'w', newline='') as fout:
with (open('input.csv', 'r', newline='') as fin,
      open('output.csv', 'w', newline='') as fout):

    reader = csv.reader(fin)
    writer = csv.writer(fout)

    # read and copy header to output
    header = next(reader)
    header.append('COMMENT')
    print(header)
    writer.writerow(header)

    for row in reader:
        node = f"The sub ID for this node is {row[1]}."
        row.append(node)
        print(row)
        writer.writerow(row)

Console output:

['GROUP_ID', 'SUB_ID', 'COMMENT']
['675233', '111', 'The sub ID for this node is 111.']
['877531', '222', 'The sub ID for this node is 222.']
['455632', '333', 'The sub ID for this node is 333.']

output.csv:

GROUP_ID,SUB_ID,COMMENT
675233,111,The sub ID for this node is 111.
877531,222,The sub ID for this node is 222.
455632,333,The sub ID for this node is 333.

CodePudding user response：

As said by the comment of Barmar.

Opening the file in w mode empties the file. Don't reopen the file in the loop, open it once at the beginning and write a row to it each time through the loop

So I see basically two options.
1- You can use open without "with", and it will be the same result

f = open("demofile.txt", "r")

instead of

with open('input.csv', 'r') as f:

2- You can store all the data you are reading and then write it in the other csv file

For example, you could create a class :

class GroupData :
   group_id = 0
   sub_id = 0
   node = ""

And then create a list of groupdata objets.
Add element in your list while reading the first file.
Then read all the elements in another loop and write in the other file.