Convert "True" and "False" to Python Boolean, But Leave Other Strings Unmodified-CodePudding

I'm working on an application using Flask-SQLAlchemy which needs to import and export data in .csv format. I have the export function working, but I've hit a snag with the import.

In short, I'm looking for a generic way to import the strings "True" and "False" as Boolean values, but any other string value as itself. The reason for this is that csv.DictWriter renders Boolean values as their string equivalents, and I need to import the resulting file back into the database.

I've tried ast.literal_eval(), but it throws a ValueError if I give it anything other than "True" or "False". I could test for the column name, but of course that would break as soon as I change my schema. I want to use the idiom

reader = csv.DictReader(fh)
...
for row in reader:
...
    contact = Contact.query.get(row['id'])
    for key, value in row.items():
        setattr(contact, key, value)

Can anyone suggest a way to achieve what I want to accomplish?

CodePudding user response：

when reading the CSV, loop through all the columns and replace the strings with the corresponding values.

csv_data = []
with open("file.csv") as f:
    csv_f = csv.reader(f)
    for row in csv_f:
        for i, col in enumerate(row):
            if col == "True":
                row[i] = True
            elif col == "False":
                row[i] = False
        csv_data.append(row)

CodePudding user response：

Why don't you use pandas to read and write? you can easily convert a column using the as_type() method

UPDATE: Assuming the test.csv is

1,True,sada
2,False,asda
3,False,afaff
4,True,agwe

read the CSV with

import pandas as pd

data = pd.read('text.csv', header=None)

Check data:

print(data.head())

  0      1      2
0  1   True   sada
1  2  False   asda
2  3  False  afaff
3  4   True   agwe

Check types:

print(data.dtypes)

0     int64
1      bool
2    object
dtype: object

Convert column to string or integers or boolean:

data[1].as_type(str)
data[1].as_type(int)
data[1].as_type(bool)

CodePudding user response：

Based on Barmar's answer above, I tried

for row in reader:
    if not row['id']:
        contact = Contact(**row)
        db.session.add(contact)
    else:
        contact = Contact.query.get(row['id'])
        for key, value in row.items():
            if value == "True":
                value = True
            if value == "False":
                value = False
            if value == '':
                value = None
            setattr(contact, key, value)

But then I realized I should be doing it in my model, in an @validates method:

class Contact(db.Model):
...
    @validates('blacklisted', 'disabled', 'is_member')
    def string_to_bool(self, key, value):
        if isinstance(value, bool):
            return value
        if value == "True":
            return True
        if value == "False":
            return False
        if value is not None and re.match("^\s*$", value) is not None:
            return None
        return value

This puts all the type-matching code in one place, where it is most relevant.