How to read different blocks from a single csv into different arrays?-CodePudding

My textfile consists of different "blocks", e.g.,

0 0 1
1 1 1
1 0 0

1 0 0 1
1 1 1 1
1 1 1 0

1 0 0 1
1 1 1 1
1 1 1 0
1 0 1 0

1 0 0
0 1 1
1 1 1

1 0 0 0 0 1
1 1 1 1 0 0
1 1 0 0 1 0
1 0 1 0 0 0

I want to read each block in a np array. I didn't find a parameter fornp.loadtxt() to read within blank lines. I guess imposing conditions at f = open('test_case_11x5.txt', 'r') for line in f: ... is slow.

Does anyone know a neat method?

CodePudding user response：

You can use the groupby function in itertools like this:

from itertools import groupby
import numpy as np

arr = []
with open('data.txt') as f_data:    
    for k, g in groupby(f_data, lambda x: x.startswith('#')):
        if not k:
            arr.append(np.array([[int(x) for x in d.split()] for d in g if len(d.strip())]))

This will yield a list of np arrays.

CodePudding user response：

Here is a working solution using re.split and a small list comprehension. I assumes the full text is first loaded in the variable text:

import re, io
import numpy as np

# text = ... ## load here your file

[np.loadtxt(io.StringIO(t)) for t in re.split('\n\n', text)]

output:

[array([[0., 0., 1.],
        [1., 1., 1.],
        [1., 0., 0.]]),
 array([[1., 0., 0., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 0.]]),
 array([[1., 0., 0., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 0.],
        [1., 0., 1., 0.]]),
 array([[1., 0., 0.],
        [0., 1., 1.],
        [1., 1., 1.]]),
 array([[1., 0., 0., 0., 0., 1.],
        [1., 1., 1., 1., 0., 0.],
        [1., 1., 0., 0., 1., 0.],
        [1., 0., 1., 0., 0., 0.]])]