find first non-null & non-empty string value-CodePudding

I was using this to find the first non null value of a string:

def get_first_non_null_values(df):
    first_non_null_values = []
    try:
        kst = df['kst'].loc[df['kst'].first_valid_index()]
        first_non_null_values.append(kst)
    except:
        kst = df['kst22'].loc[df['kst22'].first_valid_index()]
        first_non_null_values.append(kst)
    return first_non_null_values


first_non_null_values = get_first_non_null_values(df_merged)

This worked but now in my new dataset, I have some null values and some "" empty strings. How can I modify this such that I can extract the first value which is neither null not an empty string

CodePudding user response：

I think u need:

df = pd.DataFrame({'col': ['', np.nan, '', 1, 2, 3]})
print(df['col'].loc[df['col'].replace('', np.nan).first_valid_index()])

CodePudding user response：

You can use a combination of notnull/astype(bool) and idxmax:

(df['col'].notnull()&df['col'].astype(bool)).idxmax()

Example input:

>>> df = pd.DataFrame({'col': ['', float('nan'), False, None, 0, 'A', 3]})
>>> df
     col
0       
1    NaN
2  False
3   None
4      0
5      A
6      3

output: 5

null and truthy states:

     col  notnull  astype(bool)   both
0            True         False  False
1    NaN    False          True  False
2  False     True         False  False
3   None    False         False  False
4      0     True         False  False
5      A     True          True   True
6      3     True          True   True

first non empty string value:

If you're only interesting in strings that are not empty:

df['col'].str.len().gt(0).idxmax()