Python interpolate from csv file-CodePudding

I am trying to interpolate data from a csv file. The first 2 columns are the same size. I had success with same data in txt files before, this is my first attempt with a csv file. Any help would be appreciated. Sample csv file

wave,transmittance
406.06,0.384
420.06,0.398
440.85,0.416
....,.....

import numpy as np
import csv
from scipy import interpolate

# column 0  wavelength
# column 1  transmittance
# .....

with open('CCD.csv','r') as csv_file:
    next(csv_file)
    data = list(csv.reader(csv_file))

x=[column[0] for column in data]
y=[column[1] for column in data]

I = interpolate.InterpolatedUnivariateSpline(x, y)


print(x)

Output:
UFuncTypeError                 
<ipython-input-9-fa767336938e> in <module>
     12 y=[column[1] for column in data]
     13 
---> 14 I = interpolate.InterpolatedUnivariateSpline(x, y)

UFuncTypeError: ufunc 'subtract' did not contain a loop with signature matching types (dtype('<U11'), dtype('<U11')) -> dtype('<U11')

EDITED :

with open('CCD.csv','r') as csv_file:
    reader = csv.reader(csv_file)
    next(csv_file)  # discard header
    data = list(reader)
x1=[column[0] for column in data]
y1=[column[1] for column in data]

I2 = interpolate.InterpolatedUnivariateSpline(x1, y1)
z2 = I2(550)
print(z2)

UFuncTypeError                            Traceback (most recent call last)
<ipython-input-4-a8127aad6fce> in <module>
      6 y1=[column[1] for column in data]
      7 
----> 8 I2 = interpolate.InterpolatedUnivariateSpline(x1, y1)
      9 z2 = I2(550)
     10 print(z2)

UFuncTypeError: ufunc 'subtract' did not contain a loop with signature matching types (dtype('<U11'), dtype('<U11')) -> dtype('<U11')

CodePudding user response：

You're close. You need to create the reader object when you first open the file, then use the reader to iterate your rows:

import csv

data = None
with open('CCD.csv', newline='') as csv_file:
    reader = csv.reader(csv_file)
    next(csv_file)  # discard header
    data = list(reader)

x1 = [float(row[0]) for row in data]
y1 = [float(row[1]) for row in data]

print(x1)
print(y1)

and I get:

[406.06, 420.06, 440.85]
[0.384, 0.398, 0.416]

I can see how you got that, though, just coming from iterating text files line-by-line. Good try! :)

Also, you don't need to specify read-mode 'r', that's the default; but for CSV files, it's recommended that in most cases, you want newline='' so that Python leaves all newline chars (NL, LF, CRLF) as is for the CSV reader to deal with, according the CSV spec.

Alternatively, you can build x and y while you're reading the CSV:

x, y = [], []
with open('CCD.csv', newline='') as csv_file:
    reader = csv.reader(csv_file)
    next(csv_file)  # discard header
    
    for row in reader:
        x.append(float(row[0]))
        y.append(float(row[1]))

print(x)
print(y)

Same results from the prints from above.

CodePudding user response：

This is how I managed it with pandas, but I am still curious about the alternative way, without pandas

import pandas as pd

df = pd.read_csv('CCD.csv')

x = df['KAF16200wave'].to_numpy()
y = df['KAF16200transmitence'].to_numpy()

I = interpolate.InterpolatedUnivariateSpline(x, y)

z = I(550)
print("QE:",z)

QE: 0.589681398010309