Reading a Python File to EOF while performing if statments-CodePudding

I am working on creating a program to concatenate rows within a file. Each file has a header, datarows labeled DAT001 to DAT113 and a trailer. Each line of concatenated rows will have DAT001 to DAT100 and 102-113 is optional. I need to print the header, concatenating DAT001-113 and when the file finds a row with DAT001 I need to start a new line concatenating DAT001-113 again. After that is all done, I will print the trailer. I have an IF statement started but it only writes the header and skips all other logic. I apologize that this is very basic - but I am struggling with reading rows over and over again without knowing how long the file might be.

I have tried the below code but it won't read or print after the header.

import pandas as pd
destinationFile = "./destination-file.csv"
sourceFile = "./TEST.txt"
header = "RHR"
data = "DPSPOS"
beg_data = "DAT001"
data2 = "DAT002"
data3 = "DAT003"
data4 = "DAT004"
data5 = "DAT005"
data6 = "DAT006"
data7 = "DAT007"
data8 = "DAT008"
data100 = "DAT100"
data101 = "DAT101"
data102 = "DAT102"
data103 = "DAT103"
data104 = "DAT104"
data105 = "DAT105"
data106 = "DAT106"
data107 = "DAT107"
data108 = "DAT108"
data109 = "DAT109"
data110 = "DAT110"
data111 = "DAT111"
data112 = "DAT112"
data113 = "DAT113"



req_data = ''
opt101 = ''
opt102 = ''

with open(sourceFile) as Tst:
    for line in Tst.read().split("\n"):
        if header in line: 
            with open(destinationFile, "w ") as dst:
                dst.write(line)
        elif data in line:
            if beg_data in line: 
                req_data = line line line line line line line line line
                if data101 in line: 
                    opt101 = line
                    if data102 in line: 
                        opt102 = line
            new_line = pd.concat(req_data,opt101,opt102)
            with open(destinationFile, "w ") as dst:
                dst.write(new_line)
        else:
            if trailer in line:
                with open(destinationFile, "w ") as dst:
                    dst.write(line)

CodePudding user response：

Just open the output file once for the whole loop, not every time through the loop.

Check whether the line begins with DAT101. If it does, write the trailer to the current line and start a new line by printing the header.

Then for every line that begins with DAT, write it to the file in the current line.

first_line = True
with open(sourceFile) as Tst, open(destinationFile, "w ") as dst:
    for line in Tst.read().split("\n"):
        # start a new line when reading DAT101
        if line.startswith(beg_data):
            if not first_line: # need to end the current line
                dst.write(trailer   '\n')
                first_line = False
            dst.write(header)
        # copy all the lines that begin with `DAT`
        if line.startswith('DAT'):
            dst.write(line)
    # end the last line
    dst.write(trailer   '\n')

CodePudding user response：

See if the following code helps make progress. It was not tested because no Minimum Runnable Example is provided.

with open(destinationFile, "a") as dst:
    # The above will keep the file open until after all the indented code runs
    with open(sourceFile) as Tst:
        # The above will keep the file open until after all the indented code runs
        for line in Tst.read().split("\n"):
            if header in line:
                dst.write(line)
            elif data in line:
                if beg_data in line:
                    req_data = line   line   line   line   line   line   line   line   line
                    if data101 in line:
                        opt101 = line
                        if data102 in line:
                            opt102 = line
                new_line = pd.concat(req_data, opt101, opt102)
                dst.write(new_line)
        else:
            if trailer in line:
                dst.write(line)
# With is a  context manager which will automatically close the files.