I am trying to learn how to use np.arrays, and as a practice task, I am working on a program that reads data from a file and adds it to some arrays.

The function read_from_file() is supposed to return some arrays that contain the sorted datapoint so that they can be plotted easily with matplot lib.

The code I have written so far is

import numpy as np
import matplotlib.pyplot as plt

def read_from_file(filename):

    a = np.array
    b = np.array
    c = np.array
    d = np.array


    with open(filename, "r") as infile:
        infile.readline()
        for line in infile:
            infile.readline()
            delta_x = float(line[9:21].strip())
            approx = float(line[34:50].strip())
            abs_error = float(line[91:103].strip())
            relative_error = float(line[121:133].strip())
            n_ = int(line[137:].strip())

            a = np.append(delta_x)
            b = np.append(approx)
            c = np.append(abs_error)
            d = np.array(relative_error)

    return a, b, c, d

test = read_from_file("datafile.txt")

The datafile I'm trying to read from looks like this (I only care about every other line)

0.1 0.045590188541076104
delta_x: 1.000000e-01, df_approx: 4.5590188541e-01, df_exact: 5.0000000000e-01, abs_error: 4.409811e-02,           rel_error: 8.819623e-02, n=1
0.01 0.00495661575773676
delta_x: 1.000000e-02, df_approx: 4.9566157577e-01, df_exact: 5.0000000000e-01, abs_error: 4.338424e-03,           rel_error: 8.676848e-03, n=2
...................................
delta_x: 1.000000e-18, df_approx: 0.0000000000e 00, df_exact: 5.0000000000e-01, abs_error: 5.000000e-01,           rel_error: 1.000000e 00, n=18
1e-19 0.0
delta_x: 1.000000e-19, df_approx: 0.0000000000e 00, df_exact: 5.0000000000e-01, abs_error: 5.000000e-01,           rel_error: 1.000000e 00, n=19

My problem is I think I am managing to extract the data as intended (tested it out on lists etc). But when I try to add this to the np.arrays it gives me this error message:

Traceback (most recent call last):
  File "C:/(...).py", line 55, in <module>
    test = read_from_file("datafile.txt")
  File "C:/(...).py", line 48, in read_from_file
    a = np.append(delta_x)
  File "<__array_function__ internals>", line 4, in append
TypeError: _append_dispatcher() missing 1 required positional argument: 'values'

My questions are:

1. Is there anyone out there kind enough to provide me with a solution to this problem?
2. How can I add data to np.arrays (kind of like how you do with lists)
3. What is the best way of adding data to np.arrays
4. If I want to use a for-loop to iterate through an array (kind of like how you would do it with lists), what is the best way to do it?

All help is much welcomed and highly appreciated

CodePudding user response：

1. To create an empty numpy array (when you don't know the final shape), you should use np.array([])
2. np.append takes two arguments: the array to append to, and the values you want to append. You only pass one.
3. enumerate over the file so you can only parse the lines you actually need.

Try:

def read_from_file(filename):
    a = np.array([])
    b = np.array([])
    c = np.array([])
    d = np.array([])
    
    with open(filename) as infile:
        for i, line in enumerate(infile):
            if i%2 == 0:
                continue
            contents = line.split(", ")
            values = [c.split(":")[-1].strip() for c in contents]
            a = np.append(a, float(values[0]))
            b = np.append(b, float(values[1]))
            c = np.append(c, float(values[2]))
            d = np.append(d, float(values[3]))
    return a,b,c,d

a, b, c, d = read_from_file("file.txt")

>>> a
array([1.e-01, 1.e-02, 1.e-18, 1.e-19])

>>> b
array([0.45590189, 0.49566158, 0.        , 0.        ])

>>> c
array([0.5, 0.5, 0.5, 0.5])

>>> d
array([0.04409811, 0.00433842, 0.5       , 0.5       ])

CodePudding user response：

When learning to use numpy, you need to have the numpy documentation at hand, and take time to read the basics, and the help for each function that you use. Don't naively treat arrays "like lists".

def read_from_file(filename):

a = np.array

np.array is a function. Why are you assigning that to an variable? Arrays are created with expressions like a=np.array([[1,2],[3,4]]). Reread the np.array docs.

with open(filename, "r") as infile:
    infile.readline()
    for line in infile:
        infile.readline()
        delta_x = float(line[9:21].strip())
        ...

        a = np.append(delta_x)

np.append docs says it makes a copy from TWO inputs,

In [239]: x =np.append(np.array([1,2,3]), 4)
In [240]: x
Out[240]: array([1, 2, 3, 4])

It does not operate in-place like list append. The np.append docs try to make this clear. The only thing that would be clearer would be delete the function entirely!

In [241]: x=[1,2,3]
In [242]: x.append(4)
In [243]: x
Out[243]: [1, 2, 3, 4]

If you must build an array iteratively, stick with lists until the end

a = [] for _ in loop: a.append(values) arr = np.array(a)

numpy arrays can be faster than lists if they are used correctly, as documented. They are not faster if used like lists. If anything they are slower, and often don't even work.

I suspect your file can be read with np.genfromtxt, using the version of delimiter that specifies column widths.

delimiter : str, int, or sequence, optional
The string used to separate values.  By default, any consecutive
whitespaces act as delimiter.  An integer or sequence of integers
can also be provided as width(s) of each field.

Speaking of genfromtxt, that function reads a file line by line, collecting results in a list of lists (one sublist per line). It then makes the 2d array from the result at the end. It does not try to build arrays incrementally like lists.