Pandas dataframe, get the row and index for a column meeting certain conditions-CodePudding

I have the below df:

import pandas as pd
import numpy as np

output =  [['Owner', 'Database', 'Schema', 'Table', 'Column', 'Comment', 'Status'], ['', 'DEV', 'AIRFLOW', 'TASK_INSTANCE', '_LOAD_DATETIME', 'Load datetime'], ['', 'DEV', 'AIRFLOW', 'TEST', '_LOAD_FILENAME', 'load file name', 'ADDED'],['', 'DEV', 'AIRFLOW', 'TEST_TABLE', 'TEST_COL', 'COMMENT TEST'],]


df = pd.DataFrame(output[1:], columns=output[0])



query_list = []
empty_status_idx = []

for index, row in df.iterrows():
    if row['Status'] is None:
        sql = f"ALTER TABLE {row['Table']} ALTER {row['Column']} COMMENT {row['Comment']}; "
        # idx = np.where(df["Status"] is None)
        # idx = df.index[df['Status']]
        idx = df.iloc[df['Status']]
        empty_status_idx.append(idx)
        print(f'idx: {idx}')
       
        query_list.append(sql)
query_list

I see the below error with idx:

TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

What I want to see if a list of the positions of None cells:

empty_status_idx = [0, 2]

The above idx value I got from some of the answers from this stack overflow question

CodePudding user response：

You can also use a np.where method as follows:

import numpy as np
empty_status_idx = np.where(df.Status.isnull())[0].tolist()

[0, 2]

CodePudding user response：

You are overcomplicating it:

empty_status_idx = df[df['Status'].isnull()].index.tolist()

Out[65]: [0, 2]