Numpy Array is not copying over all values of the array to csv file-CodePudding

I'm having trouble with the formatting of my array when it it saved to a csv file, each time I run the program I get a different result, sometimes the string "creditCard" is not displayed in the spread sheet but all other details are. and sometimes it is. is there something obvious in my code that is causing this flakiness? I have attached the relevant code

def start():


dtype = [('Name', (np.str_, 10 )), ('Number of uses', np.int32), ('Card Value', np.float64)]
defaultCardArray = np.array([("creditCard",4,200)],dtype=dtype)
np.savetxt('creditCards.csv', defaultCardArray, delimiter=',', fmt=['%s' , '%d', '%f'],
           header='Name,Number of Uses, Card Value', comments='')e here

after the file is created I also have the following code to add each card there after. the default card loads in fine with the file creation but the following card array is what Im having issue with there is code to ask the user for input and that saves to the variables without error so I haven't included that here

cardArray = np.array([creditCard,itemLimit,cardValueInput]).reshape(1,3)
file = open(r'creditCards.csv', 'at', newline ='') 
with file:     
    write = csv.writer(file) 
    write.writerows(cardArray) 
    file.close()

image of the output I am getting in the csv

CodePudding user response：

For you code, I think when you create the new cardArray you don't specify the type as you did for the initial array, this could be an issue.

Anyways, numpy might not be the best library to use here.
For this kind of data I would use pandas:

import pandas as pd

header = ['Name', 'Number of Uses', 'Card Value']

def start():
# create default dataset
  default_card_df =  pd.DataFrame(["creditCard",4,200], columns=header)
  default_card_df.to_csv('creditCards.csv', sep=",")

def add_card(name, itemLimit, cardValueInput):
# read csv, add new data, save to csv
  df = pd.read_csv('creaditCards.csv', sep=",")
  new_df = pd.DataFrame([name, itemLimit, cardValueInput], columns=header)
  updated_df = pd.concat(df, new_df)
  updated_df.to_csv('creditCards.csv', sep=",")

CodePudding user response：

Your initial write:

In [281]: np.savetxt('creditCards.csv', defaultCardArray, delimiter=',', fmt=['%
     ...: s' , '%d', '%f'],
     ...:            header='Name,Number of Uses, Card Value', comments='')

produces:

In [282]: cat creditCards.csv
Name,Number of Uses, Card Value
creditCard,4,200.000000

We can add more entries to that file if it's opened in append mode:

In [284]: with open('creditCards.csv','a') as f:
     ...:     np.savetxt(f, defaultCardArray, delimiter=',', fmt=['%s' , '%d', '
     ...: %f'])
     ...: 
In [285]: cat creditCards.csv
Name,Number of Uses, Card Value
creditCard,4,200.000000
creditCard,4,200.000000

To continue to use savetxt with the same fmt, you should be creating arrays with original dtype.

In [286]: newcards = np.zeros(3,dtype=dtype)
In [287]: newcards
Out[287]: 
array([('', 0, 0.), ('', 0, 0.), ('', 0, 0.)],
      dtype=[('Name', '<U10'), ('Number of uses', '<i4'), ('Card Value', '<f8')])

In [289]: newcards['Name']='foobar'
In [290]: newcards['Card Value']=np.arange(10,40,10)
In [291]: newcards
Out[291]: 
array([('foobar', 0, 10.), ('foobar', 0, 20.), ('foobar', 0, 30.)],
      dtype=[('Name', '<U10'), ('Number of uses', '<i4'), ('Card Value', '<f8')])

In [292]: with open('creditCards.csv','a') as f:
     ...:     np.savetxt(f, newcards, delimiter=',', fmt=['%s' , '%d', '%f'])
     ...: 

In [293]: cat creditCards.csv
Name,Number of Uses, Card Value
creditCard,4,200.000000
creditCard,4,200.000000
foobar,0,10.000000
foobar,0,20.000000
foobar,0,30.000000