Saving big data in csv file-CodePudding

I am trying to save a large matrix, 1000x1000 which follows log-normal distribution. But the saved file turns out to be empty. What am I doing incorrectly here?

import numpy as np
import csv 

with open('Radius.csv', 'w') as f:
    shape = 1000,1000
    zmin, zmax = 0.2,0.8
    n = np.prod(shape) 
    zc = np.array([]) 
    while True: 
        z = np.random.lognormal(mean=0.2, sigma=0.5, size=n * 100) 
        z = z[(zmin <= z) & (z < zmax)] 
        z = np.r_[zc, z] 
        if len(z) >= n: 
            break 
        inv_r = z[:n].reshape(shape) 
        print("1/r =",[inv_r])
        writer = csv.writer(f)
        writer.writerows(zip(1,[inv_r]))

CodePudding user response：

It has to do with the way you are writing to rows, the zip function takes in two iterables, you passed in an int and an iterable [list] the while loop also only ever will go through once as it stands. If you run this:

import numpy as np
import csv 

with open('Radius.csv', 'w ') as f:
    shape = 1000,1000
    zmin, zmax = 0.2,0.8
    n = np.prod(shape) 
    zc = np.array([]) 
    z = np.random.lognormal(mean=0.2, sigma=0.5, size=n * 100) 
    z = z[(zmin <= z) & (z < zmax)] 
    z = np.r_[zc, z] 
    inv_r = z[:n].reshape(shape) 
    print("1/r =",[inv_r])
    writer = csv.writer(f)
    writer.writerows(inv_r)

it will at least log to the csv, definitely check your zip function to make sure it does what you want it to!