CSV append in loop is overwriting csv file (except for headers)-CodePudding

I am trying to pull 26 system names and 8 data point each for some hardware via API's.

The console is outputting the correct values from the last print statement. However I am getting only the last system and 8 values

import requests
import json
import urllib3
import csv
import os


urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

#### open text file with list of arrays
with open('arrays.txt', 'r') as f:
     for line in f:
         #print(line.strip())
         data = line.strip()
         s1 = str (data)
         arrayPortUrl = "https://{}/api/1.17/port".format(str(s1))
         #### authentication token request
         token_url = "https://{}/api/1.17/auth/apitoken".format(str(s1))
         payload = {'username': 'user', 'password': 'password'}
         headers = {'Content-Type': 'application/json'}
         response = requests.request("POST", token_url, data=json.dumps(payload),
         headers=headers, verify=False, timeout=3000)
         token = response.text
         #print(response.text)

         #### Session setup
         # session request with token
         session = requests.Session()
         sessionUrl = "https://{}/api/1.17/auth/session".format(str(s1))
         payload = response.text
         response = session.request("POST", sessionUrl, data=payload, headers=headers,
                    verify=False, timeout=3000)
        #print(response.text)

        ### port information retrieval
        baseUrl = "https://{}/api/1.17/port".format(str(s1))
        response = session.request("GET", baseUrl, headers=headers, verify=False,
                   timeout=3000)
        content = response.content
        info = json.loads(content)

        ###### loop statement to write all array names, port names, and wwn
        header = ['Array', 'FCS_port', 'WWN']
        with open('ports.csv', 'w', encoding='UTF8',newline="") as f:
        writer = csv.writer(f)
        writer.writerow(header)
        for i in info:
            portI = (s1, i['name'], i['wwn'])
            with open('ports.csv', 'a', newline='') as p:
                writer = csv.writer(p)
                writer.writerow(portI)
                print(portI)

Print output state show the correct list for each line 26 rows x8. I cannot post the output as it contains confidential data.

File output is only the last system and 8 wwn values.

CSV file in Excel:

screenshot

CodePudding user response：

You have open('ports.csv', 'w') inside a for-loop (the outer for-loop).

This overwrites the file on each iteration of the loop, so you are only getting the information for the last iteration (system) saved there.

You need to open this file before the loop, write everything in it and the close it after the loop.

CodePudding user response：

in your main for loop every time it arives to :

with open('ports.csv', 'w', encoding='UTF8',newline="") as f:

your csv file will be empty so this line that names the headings must execute only one time

to solve the problem you should take out this code from the loop and it will be :

import requests
import json
import urllib3
import csv
import os


urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

#### create ports.csv file and name the headings
header = ['Array', 'FCS_port', 'WWN']
with open('ports.csv', 'w', encoding='UTF8',newline="") as f:
    writer = csv.writer(f)
    writer.writerow(header)

#### open text file with list of arrays
with open('arrays.txt', 'r') as f:
     for line in f:
         #print(line.strip())
         data = line.strip()
         s1 = str (data)
         arrayPortUrl = "https://{}/api/1.17/port".format(str(s1))
         #### authentication token request
         token_url = "https://{}/api/1.17/auth/apitoken".format(str(s1))
         payload = {'username': 'user', 'password': 'password'}
         headers = {'Content-Type': 'application/json'}
         response = requests.request("POST", token_url, data=json.dumps(payload),
         headers=headers, verify=False, timeout=3000)
         token = response.text
         #print(response.text)

         #### Session setup
         # session request with token
         session = requests.Session()
         sessionUrl = "https://{}/api/1.17/auth/session".format(str(s1))
         payload = response.text
         response = session.request("POST", sessionUrl, data=payload, headers=headers,
                    verify=False, timeout=3000)
        #print(response.text)

        ### port information retrieval
        baseUrl = "https://{}/api/1.17/port".format(str(s1))
        response = session.request("GET", baseUrl, headers=headers, verify=False,
                   timeout=3000)
        content = response.content
        info = json.loads(content)

        ###### loop statement to write all array names, port names, and wwn
        for i in info:
            portI = (s1, i['name'], i['wwn'])
            with open('ports.csv', 'a', newline='') as p:
                writer = csv.writer(p)
                writer.writerow(portI)
                print(portI)