Is there a way to writerows then immediately modify and writerow afterwards?-CodePudding

So I have a csv that I am trying to modify by inserting rows, but also looping through an iteration to append additional columns to the CSV prior to closing the CSV. I have tried various different ways but it seems as if once you have writerows function completed, the next step doesn't process any of the writerows. Is there a way of doing this, or would I have to save the writerows result, reopen the CSV file then complete the appending? I have inserted the rows into a CSV, then commented out the output.writerows(rows) and the for loop does work in that case. But, with writerows inserted, it no longer works as I would want it too as it skips anything that was written during the function.

Here is some sample code I've been working with:

import csv
import itertools
import uuid
from datetime import datetime

rows = [[1000000, 'Test', 'Test', 179, 1111, 222, 5, 'Test', 'Test', 1111, 'Test', 222, 'Test', 1111, 22222, 'Test', '', 1, 2],
        [1000000, 'Test', 'Test', 179, 1111, 222, 5, 'Test', 'Test', 1111, 'Test', 222, 'Test', 1111, 22222, 'Test', '', 1, 2],
        [1000000, 'Test', 'Test', 179, 1111, 222, 5, 'Test', 'Test', 1111, 'Test', 222, 'Test', 1111, 22222, 'Test', '', 1, 2],
        [1000000, 'Test', 'Test', 179, 1111, 222, 5, 'Test', 'Test', 1111, 'Test', 222, 'Test', 1111, 22222, 'Test', '', 1, 2],
        [1000000, 'Test', 'Test', 179, 1111, 222, 5, 'Test', 'Test', 1111, 'Test', 222, 'Test', 1111, 22222, 'Test', '', 1, 2],
        [1000000, 'Test', 'Test', 179, 1111, 222, 5, 'Test', 'Test', 1111, 'Test', 222, 'Test', 1111, 22222, 'Test', '', 1, 2],
        [1000000, 'Test', 'Test', 179, 1111, 222, 5, 'Test', 'Test', 1111, 'Test', 222, 'Test', 1111, 22222, 'Test', '', 1, 2],
        [1000000, 'Test', 'Test', 179, 1111, 222, 5, 'Test', 'Test', 1111, 'Test', 222, 'Test', 1111, 22222, 'Test', '', 1, 2],
        [1000000, 'Test', 'Test', 179, 1111, 222, 5, 'Test', 'Test', 1111, 'Test', 222, 'Test', 1111, 22222, 'Test', '', 1, 2]]


def _sanitize_report(source_file, dest_file, append_timestamp, timestamp_format, append_filename, file,
                        append_id):
    with open(source_file, 'r') as report:
        reader = csv.reader(report)
        with open(dest_file, 'w') as output_file:
            data_rows = 0
            output = csv.writer(output_file)
            output.writerows(rows)
            for index, row in enumerate(reader):
                if append_timestamp and append_filename and append_id:
                    output.writerow(itertools.chain([datetime.today().strftime(timestamp_format)],
                                                    [str(file).replace(file[0:3], '')],
                                                    [uuid.uuid4()]))
                elif append_timestamp and append_filename:
                    output.writerow(itertools.chain([datetime.today().strftime(timestamp_format)],
                                                    [str(file).replace(file[0:3], '')]))
                elif append_timestamp and append_id:
                    output.writerow(itertools.chain([datetime.today().strftime(timestamp_format)], [uuid.uuid4()]))
                elif append_filename and append_id:
                    output.writerow(itertools.chain([str(file).replace(file[0:3], '')], [uuid.uuid4()]))
                elif append_timestamp:
                    output.writerow(itertools.chain( [datetime.today().strftime(timestamp_format)]))
                elif append_filename:
                    output.writerow(itertools.chain([str(file).replace(file[0:3], '')]))
                elif append_id:
                    output.writerow(itertools.chain([uuid.uuid4()]))
                else:
                    output.writerow([value.replace('(not set)', '') for value in row])
                data_rows  = 1


_sanitize_report('test.csv', 'test_'   str(uuid.uuid4())   '.csv',
                    append_timestamp=True, timestamp_format=datetime.utcnow().isoformat()   'Z',
                    append_filename=True, file='123Test_success', append_id=True)

The End result that I'm looking for is for the user to decide what is needed to be appended to the results and save the file into a new location with everything that is needed. Is this possible to do in a single function or would it have to be broken up? I have tried multiple different ways with no success and I've been at it for about a week now.

Any help on this would be incredibly appreciated!

CodePudding user response：

What happens when you call writerows() followed by some number of individual calls to writerow()?

rows = [
    ["r1c1", "r1c2", "r1c3"],
    ["r2c1", "r2c2", "r2c3"],
    ["r3c1", "r3c2", "r3c3"],
]

with open("output.csv", "w", newline="") as f_out:
    writer = csv.writer(f_out)
    writer.writerows(rows)
    writer.writerow(["r4c1", "r4c2", "r4c3"])
    writer.writerow(["r5c1", "r5c2", "r5c3"])

This happens:

r1c1,r1c2,r1c3
r2c1,r2c2,r2c3
r3c1,r3c2,r3c3
r4c1,r4c2,r4c3
r5c1,r5c2,r5c3

It's the same thing that would have happened if the individual rows had been in the list when you called writerows(), or the same if you iterated the list of rows calling writerow().

writerow() writes a single row, and writerows() writes a list of rows.

If you want to selectively tack on some columns to your data, you need to add the columns to the rows before you call either writerow() or writerows(). Here's how it looks with writerow():

...
ts = datetime.today()
file_id = "foo_1"
for row in rows:
    if append_timestamp:
        row.append(ts)
    if append_id:
        row.append(file_id)
    ...
    writer.writerow(row)