Simultaneously Open, write and save multiple files separately-CodePudding

I created a python code to open multiple files and edit them and finally save each one separately

the first out file is ok, but after that, the size of the other files increases one by one.

import json
import os
from datetime import datetime
os.chdir('C:\\Users\\user1\\Desktop\\Test')
list_Files = []

date = datetime.now()
result = []
counter = 0
cnt = 0
for single_file in os.listdir('C:\\Users\\user1\\Desktop\\Test') :
    list_Files.append(single_file)
    print(single_file)
    with open(single_file, encoding='utf-8') as fn:
        for line in fn.readlines():
            counter =1
            result.append(json.loads(line))                  
    cnt =1
    print(cnt)
    json_file = 'C:\\Users\\user1\\Desktop\\Out\\tst_' f"{str(single_file)}{cnt}"'.json'
    with open(json_file, 'w') as outfile:
        json.dump(result, outfile)

CodePudding user response：

You initialize the results array before looping over all your files. While looping over your files you append the results array, making this array longer result.append(json.loads(line)). Then, within your file loop you write the results array to your json file. Therefore, each new json file receives a longer results array, and will therefore have a larger file size.

If you initialize/reset your results array within your file loop you be good. See the example code below.

import json
import os
from datetime import datetime
os.chdir('C:\\Users\\user1\\Desktop\\Test')
list_Files = []

date = datetime.now()
# result = [] removed the results array here
counter = 0
cnt = 0
for single_file in os.listdir('C:\\Users\\user1\\Desktop\\Test') :
    result = [] # initialize/reset the results array here
    list_Files.append(single_file)
    print(single_file)
    with open(single_file, encoding='utf-8') as fn:
        for line in fn.readlines():
            counter =1
            result.append(json.loads(line))                  
    cnt =1
    print(cnt)
    json_file = 'C:\\Users\\user1\\Desktop\\Out\\tst_' f"{str(single_file)}{cnt}"'.json'
    with open(json_file, 'w') as outfile:
        json.dump(result, outfile)