Pyomo DataPortal Not Reading in Multi-Column CSV Properly-CodePudding

I am trying to read in a csv with multiple columns using the Pyomo DataPortal function data.load to initialize an index i but am running into an issue where I cannot select for the column I want using the select argument.

Here is the code I am running.

from pyomo.environ import *
model = AbstractModel()
model.i = Set()

data = DataPortal()
data.load(filename='fips.csv',select='fips',set=model.i)

The fips.csv file has multiple columns with FIPS code details. I'm trying to select the column called fips.

The error I am getting is the following.

Model declaration 'f' not found in returned query columns
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Users/cwayner/PycharmProjects/SeniorThesis/pyomo/dataportal/DataPortal.py", line 164, in load
    self._data_manager.read()
  File "/Users/cwayner/PycharmProjects/SeniorThesis/pyomo/dataportal/plugins/csv_table.py", line 58, in read
    self._set_data(tmp[0], tmp[1:])
  File "/Users/cwayner/PycharmProjects/SeniorThesis/pyomo/dataportal/TableData.py", line 117, in _set_data
    header_index.append(headers.index(str(i)))
ValueError: 'f' is not in list

What is confusing about this to me is that this error includes just the first letter (f) of the column that I want to select (fips). I don't know what could be causing this in Python.

I have tried loading the data in using Pandas and all the column headers appeared to be loading correctly, but I want to figure out why it isn't working using the data.load feature. I am able to load in a one-column CSV just fine (which is a simple workaround where I split my CSV into multiple one-column CSVs), but I'd like to get the select function working for multi-column CSVs for simplicity and ease of use.

CodePudding user response：

If you look closely at the documentation for select it says the argument should be a list or tuple. So, internally to that function, the argument is "iterated". And, strings are iterable so the first "f" is being peeled off.

Solution: If you just have one column name, put it into a 1-element list or 1-element tuple. Note: you need the trailing comma in a 1-element tuple or it is just interpreted as parenthesis. Try this:

data.load(filename='fips.csv',select=('fips',),set=model.i)

Also in that same section of documentation, there is a note about loading multiple parameters at once, which might be helpful to you.