Home > Software engineering >  In python, with pandas, change several columns at once and modify some values (of these columns) to
In python, with pandas, change several columns at once and modify some values (of these columns) to

Time:02-13

After looking at SO, I (think I) could force some columns to become numeric and transform to missing values outside a predefined range. (I think the code is ok).

change values

However, when I try the same with the real data, I'm receiving the following error typeError: Invalid comparison between dtype=datetime64[ns] and int

error

The simulated data and code: (working)

#dataframe
df2 = pd.DataFrame({"var_1": [2,"a"], "var_2": [5,7], "var_3": [1,np.NaN], "other_1": [1,9]})
df2
#modify any variable with "var" 
df2.loc[:,df2.columns.str.contains("var")] = df2.apply(pd.to_numeric, errors='coerce') 
df2.loc[:,df2.columns.str.contains("var")] = df2.mask((df2 < 1) | (df2 > 5))
df2

The quasi-real dataset and the code (not working):

from numpy import nan
df = pd.DataFrame.from_dict(
{'aut_a_p1_r1': {105: 3, 194: 4, 93: 3, 297: 3, 131: 4}, 'aut_a_p1_r2': {105: 4.0, 194: 5.0, 93: 3.0, 297: 5.0, 131: 4.0}, 'aut_a_p1_r3': {105: 5.0, 194: 5.0, 93: 4.0, 297: 3.0, 131: 5.0}, 'aut_a_p1_r4': {105: 2, 194: 1, 93: 3, 297: 4, 131: 3}, 'aut_a_p1_r5': {105: 4, 194: 4, 93: 4, 297: 1, 131: 4}, 'aut_a_p1_r6': {105: 4, 194: 5, 93: 4, 297: 3, 131: 5}, 'aut_a_p1_r7': {105: 5, 194: 5, 93: 3, 297: 3, 131: 5}, 'aut_a_p1_r8': {105: 3, 194: 2, 93: 3, 297: 4, 131: 2}, 'aut_a_p1_r9': {105: 4, 194: 4, 93: -1.5, 297: 2, 131: 5}, 'aut_a_p1_r10': {105: 4, 194: 5, 93: -1.4, 297: 4, 131: 4}, 'aut_a_p1_r11': {105: 5, 194: 5, 93: 5, 297: 3, 131: 4}, 'aut_a_p1_r12': {105: 3.0, 194: 4.0, 93: 4.0, 297: 4.0, 131: 5.0}, 'aut_a_p1_r13': {105: 4, 194: 5, 93: 3, 297: 2, 131: 5}, 'aut_a_p1_r14': {105: 4, 194: 4, 93: 4, 297: 4, 131: 4}, 'aut_a_p1_r15': {105: 3.0, 194: 4.0, 93: 5.0, 297: 3.0, 131: 5.0}, 'aut_a_p2_r1': {105: 5, 194: 5, 93: 4, 297: 4, 131: 5}, 'aut_a_p2_r2': {105: 4, 194: 3, 93: 2, 297: 1, 131: 1}, 'aut_a_p2_r3': {105: 4, 194: 5, 93: 5, 297: 4, 131: 4}, 'aut_a_p2_r4': {105: 3, 194: 4, 93: 4, 297: 2, 131: 3}, 'aut_a_p2_r5': {105: 4, 194: 5, 93: 4, 297: 4, 131: 4}, 'aut_a_p2_r6': {105: 3, 194: 4, 93: 2, 297: 2, 131: 3}, 'aut_a_p2_r7': {105: 4, 194: 5, 93: 4, 297: 2, 131: 3}, 'aut_a_p2_r8': {105: 4, 194: 4, 93: 3, 297: 2, 131: 2}, 'aut_a_p2_r9': {105: 3, 194: 5, 93: 3, 297: 2, 131: 5}, 'aut_a_p2_r10': {105: 3, 194: 4, 93: 3, 297: 2, 131: 5}, 'aut_a_p2_r11': {105: 3, 194: 4, 93: 4, 297: 2, 131: 2}, 'aut_a_p2_r12': {105: 4, 194: 5, 93: 4, 297: 3, 131: 2}, 'aut_a_p2_r13': {105: 4, 194: 3, 93: 3, 297: 4, 131: 4}, 'aut_a_p2_r14': {105: 4, 194: 4, 93: 4, 297: 2, 131: 4}, 'aut_a_p2_r15': {105: 4, 194: 4, 93: 5, 297: 4, 131: 5}, 'gpe_a_p1_r1': {105: 5, 194: 5, 93: 3, 297: 2, 131: 5}, 'gpe_a_p1_r2': {105: 4, 194: 4, 93: 3, 297: 2, 131: 4}, 'gpe_a_p1_r3': {105: 2, 194: 1, 93: 3, 297: 4, 131: 1}, 'gpe_a_p1_r4': {105: 4, 194: 5, 93: 3, 297: 2, 131: 3}, 'gpe_a_p1_r5': {105: 5, 194: 5, 93: 5, 297: 4, 131: 4}, 'gpe_a_p1_r6': {105: 4, 194: 5, 93: 5, 297: 2, 131: 3}, 'gpe_a_p1_r7': {105: 3, 194: 4, 93: 3, 297: 2, 131: 2}, 'gpe_a_p1_r8': {105: 3, 194: 4, 93: 4, 297: 2, 131: 4}, 'gpe_a_p1_r9': {105: 4, 194: 5, 93: 5, 297: 3, 131: 5}, 'gpe_a_p1_r10': {105: 3, 194: 3, 93: 4, 297: 1, 131: 1}, 'gpe_a_p1_r11': {105: 3, 194: 4, 93: 3, 297: 3, 131: 2}, 'gpe_a_p1_r12': {105: 3, 194: 4, 93: 4, 297: 2, 131: 5}, 'gpe_a_p1_r13': {105: 3, 194: 4, 93: 3, 297: 2, 131: 3}, 'gpe_a_p1_r14': {105: 3, 194: 2, 93: 2, 297: 5, 131: 1}, 'gpe_a_p1_r15': {105: 4, 194: 5, 93: 3, 297: 1, 131: 3}, 'gpe_a_p2_r1': {105: 3.0, 194: 5.0, 93: 4.0, 297: 2.0, 131: 4.0}, 'gpe_a_p2_r2': {105: 3.0, 194: 4.0, 93: 2.0, 297: 1.0, 131: 4.0}, 'gpe_a_p2_r3': {105: 3, 194: 4, 93: 2, 297: 1, 131: 'BLANK'}, 'gpe_a_p2_r4': {105: 3.0, 194: 5.0, 93: 4.0, 297: 4.0, 131: 4.0}, 'gpe_a_p2_r5': {105: 4.0, 194: 5.0, 93: 5.0, 297: 5.0, 131: 5.0}, 'gpe_a_p2_r6': {105: 3.0, 194: 4.0, 93: 2.0, 297: 1.0, 131: 3.0}, 'gpe_a_p2_r7': {105: 4.0, 194: 5.0, 93: 4.0, 297: 2.0, 131: 4.0}, 'gpe_a_p2_r8': {105: 4.0, 194: 5.0, 93: 4.0, 297: 5.0, 131: 4.0}, 'gpe_a_p2_r9': {105: 3.0, 194: 5.0, 93: 2.0, 297: 1.0, 131: 3.0}, 'gpe_a_p2_r10': {105: 4.0, 194: 3.0, 93: 3.0, 297: 4.0, 131: 5.0}, 'gpe_a_p2_r11': {105: 4.0, 194: 4.0, 93: 2.0, 297: 1.0, 131: 2.0}, 'gpe_a_p2_r12': {105: 4, 194: 4, 93: 3, 297: 2, 131: 2}, 'gpe_a_p2_r13': {105: 4.0, 194: 5.0, 93: 4.0, 297: 5.0, 131: 5.0}, 'gpe_a_p2_r14': {105: 4.0, 194: 5.0, 93: 4.0, 297: 2.0, 131: 4.0}, 'gpe_a_p2_r15': {105: 3.0, 194: 5.0, 93: 4.0, 297: 2.0, 131: 4.0}, 'tdr_a_p1_r1': {105: 4, 194: 4, 93: 4, 297: 4, 131: 1}, 'tdr_a_p1_r2': {105: 4, 194: 5, 93: 4, 297: 4, 131: 5}, 'tdr_a_p1_r3': {105: 3, 194: 4, 93: 3, 297: 4, 131: 3}, 'tdr_a_p1_r4': {105: 4, 194: 5, 93: 3, 297: 4, 131: 4}, 'tdr_a_p1_r5': {105: 3, 194: 5, 93: 4, 297: 4, 131: 3}, 'tdr_a_p1_r6': {105: 5, 194: 5, 93: 3, 297: 4, 131: 4}, 'tdr_a_p1_r7': {105: 4, 194: 5, 93: 4, 297: 3, 131: 3}, 'tdr_a_p1_r8': {105: 5, 194: 5, 93: 3, 297: 3, 131: 4}, 'tdr_a_p1_r9': {105: 2, 194: 3, 93: 2, 297: 4, 131: 1}, 'tdr_a_p1_r10': {105: 3, 194: 4, 93: 3, 297: 2, 131: 3}, 'tdr_a_p1_r11': {105: 4, 194: 5, 93: 4, 297: 3, 131: 3}, 'tdr_a_p1_r12': {105: 3, 194: 5, 93: 3, 297: 4, 131: 4}, 'tdr_a_p1_r13': {105: 5, 194: 5, 93: 4, 297: 3, 131: 5}, 'tdr_a_p1_r14': {105: 4, 194: 4, 93: 3, 297: 3, 131: 4}, 'tdr_a_p1_r15': {105: 3, 194: 4, 93: 4, 297: 3, 131: 4}, 'tdr_a_p2_r1': {105: 4, 194: 5, 93: 4, 297: 3, 131: 4}, 'tdr_a_p2_r2': {105: 4, 194: 5, 93: 4, 297: 5, 131: 5}, 'tdr_a_p2_r3': {105: 4, 194: 5, 93: 4, 297: 3, 131: 3}, 'tdr_a_p2_r4': {105: 4, 194: 5, 93: 4, 297: 4, 131: 5}, 'tdr_a_p2_r5': {105: 4, 194: 5, 93: 4, 297: 2, 131: 4}, 'tdr_a_p2_r6': {105: 4, 194: 4, 93: 4, 297: 4, 131: 5}, 'tdr_a_p2_r7': {105: 4, 194: 5, 93: 3, 297: 3, 131: 5}, 'tdr_a_p2_r8': {105: 4, 194: 5, 93: 4, 297: -2.4, 131: 4}, 'tdr_a_p2_r9': {105: 4, 194: 3, 93: 3, 297: 2, 131: 5}, 'tdr_a_p2_r10': {105: 3, 194: 4, 93: 3, 297: 3, 131: 4}, 'tdr_a_p2_r11': {105: 3, 194: 4, 93: 3, 297: 4, 131: 5}, 'tdr_a_p2_r12': {105: 4, 194: 4, 93: 4, 297: 4, 131: 5}, 'tdr_a_p2_r13': {105: 4, 194: 4, 93: 4, 297: 4, 131: 5}, 'tdr_a_p2_r14': {105: 4, 194: 5, 93: 4, 297: 4, 131: 4}, 'tdr_a_p2_r15': {105: 4, 194: 5, 93: 4, 297: 4, 131: 5}, 'cso_a_p1_r1': {105: 3.0, 194: 1.0, 93: 2.0, 297: nan, 131: 3.0}, 'cso_a_p1_r2': {105: 3, 194: 4, 93: 3, 297: nan, 131: 2}, 'cso_a_p1_r3': {105: 4.0, 194: 5.0, 93: 5.0, 297: nan, 131: 5.0}, 'cso_a_p1_r4': {105: 4, 194: 5, 93: 2, 297: nan, 131: 4}, 'cso_a_p1_r5': {105: 4.0, 194: 5.0, 93: 4.0, 297: nan, 131: 3.0}, 'cso_a_p1_r6': {105: 4, 194: 4, 93: 4, 297: nan, 131: 4}, 'cso_a_p1_r7': {105: 4, 194: 5, 93: 4, 297: nan, 131: 5}, 'cso_a_p1_r8': {105: 4, 194: 5, 93: 4, 297: nan, 131: 4}, 'cso_a_p1_r9': {105: 4.0, 194: 3.0, 93: 4.0, 297: nan, 131: 5.0}, 'cso_a_p1_r10': {105: 4.0, 194: 5.0, 93: 5.0, 297: nan, 131: 5.0}, 'cso_a_p1_r11': {105: 4.0, 194: 5.0, 93: 4.0, 297: nan, 131: 5.0}, 'cso_a_p1_r12': {105: 4.0, 194: 5.0, 93: 5.0, 297: nan, 131: 4.0}, 'cso_a_p1_r13': {105: 4.0, 194: 2.0, 93: 4.0, 297: nan, 131: 4.0}, 'cso_a_p1_r14': {105: -3.4, 194: 3.0, 93: 2.0, 297: nan, 131: 3.0}, 'cso_a_p1_r15': {105: 4.0, 194: 5.0, 93: 4.0, 297: nan, 131: 5.0}, 'cso_a_p2_r1': {105: 4, 194: 5, 93: 4, 297: 5, 131: 5}, 'cso_a_p2_r2': {105: 4.0, 194: 4.0, 93: 3.0, 297: 5.0, 131: 4.0}, 'cso_a_p2_r3': {105: 4.0, 194: 4.0, 93: 4.0, 297: 4.0, 131: 5.0}, 'cso_a_p2_r4': {105: 3, 194: 5, 93: 4, 297: 4, 131: 4}, 'cso_a_p2_r5': {105: 4.0, 194: 5.0, 93: 4.0, 297: 5.0, 131: 5.0}, 'cso_a_p2_r6': {105: 4.0, 194: 5.0, 93: 4.0, 297: 5.0, 131: 5.0}, 'cso_a_p2_r7': {105: 4, 194: 5, 93: 4, 297: 5, 131: 5}, 'cso_a_p2_r8': {105: 3, 194: 5, 93: 4, 297: 4, 131: 5}, 'cso_a_p2_r9': {105: 4.0, 194: 3.0, 93: 4.0, 297: 5.0, 131: 5.0}, 'cso_a_p2_r10': {105: 4, 194: 2, 93: 4, 297: 3, 131: 5}, 'cso_a_p2_r11': {105: 4.0, 194: 4.0, 93: 4.0, 297: 5.0, 131: 5.0}, 'cso_a_p2_r12': {105: 4.0, 194: 4.0, 93: 4.0, 297: 5.0, 131: 5.0}, 'cso_a_p2_r13': {105: 4, 194: 4, 93: 3, 297: 5, 131: 5}, 'cso_a_p2_r14': {105: 4, 194: 5, 93: -3.4, 297: 5, 131: 4}, 'cso_a_p2_r15': {105: 4.0, 194: 5.0, 93: 4.0, 297: 4.0, 131: 4.0}, 'rin_a_p1_r1': {105: 3.0, 194: 5.0, 93: 4.0, 297: 2.0, 131: 4.0}, 'rin_a_p1_r2': {105: 3.0, 194: 5.0, 93: 3.0, 297: 2.0, 131: 2.0}, 'rin_a_p1_r3': {105: 3.0, 194: 5.0, 93: 4.0, 297: 2.0, 131: 5.0}, 'rin_a_p1_r4': {105: 2, 194: 4, 93: 1, 297: 4, 131: 2}, 'rin_a_p1_r5': {105: 3.0, 194: 5.0, 93: 3.0, 297: 4.0, 131: 2.0}, 'rin_a_p1_r6': {105: 3, 194: 3, 93: 4, 297: 2, 131: 5}, 'rin_a_p1_r7': {105: 3.0, 194: 5.0, 93: 3.0, 297: 4.0, 131: 5.0}, 'rin_a_p1_r8': {105: 3.0, 194: 4.0, 93: 3.0, 297: 4.0, 131: 5.0}, 'rin_a_p1_r9': {105: 3.0, 194: 4.0, 93: 3.0, 297: 4.0, 131: 2.0}, 'rin_a_p1_r10': {105: 3, 194: 5, 93: 4, 297: 3, 131: 5}, 'rin_a_p1_r11': {105: 3, 194: 5, 93: 3, 297: 3, 131: 4}, 'rin_a_p1_r12': {105: 3, 194: 4, 93: 2, 297: 1, 131: 3}, 'rin_a_p1_r13': {105: 3.0, 194: 4.0, 93: 3.0, 297: 5.0, 131: 3.0}, 'rin_a_p1_r14': {105: 3.0, 194: 5.0, 93: 4.0, 297: 4.0, 131: 5.0}, 'rin_a_p1_r15': {105: 3.0, 194: 5.0, 93: 2.0, 297: 4.0, 131: 1.0}, 'rin_a_p2_r1': {105: 4, 194: 5, 93: 5, 297: nan, 131: 5}, 'rin_a_p2_r2': {105: 4, 194: 5, 93: 4, 297: nan, 131: 5}, 'rin_a_p2_r3': {105: 4, 194: 4, 93: 4, 297: nan, 131: 5}, 'rin_a_p2_r4': {105: 4, 194: 2, 93: 4, 297: nan, 131: 4}, 'rin_a_p2_r5': {105: 4, 194: 3, 93: 4, 297: nan, 131: 4}, 'rin_a_p2_r6': {105: 3, 194: 4, 93: 3, 297: nan, 131: 3}, 'rin_a_p2_r7': {105: 3, 194: 4, 93: 3, 297: nan, 131: 5}, 'rin_a_p2_r8': {105: 4, 194: 5, 93: 4, 297: nan, 131: 4}, 'rin_a_p2_r9': {105: 4, 194: 4, 93: 4, 297: nan, 131: 4}, 'rin_a_p2_r10': {105: 4, 194: 5, 93: 4, 297: nan, 131: 3}, 'rin_a_p2_r11': {105: 4, 194: 4, 93: 4, 297: nan, 131: 5}, 'rin_a_p2_r12': {105: 3, 194: 4, 93: 4, 297: nan, 131: 3}, 'rin_a_p2_r13': {105: 4, 194: 5, 93: 4, 297: nan, 131: 5}, 'rin_a_p2_r14': {105: 4, 194: 4, 93: 3, 297: nan, 131: 5}, 'rin_a_p2_r15': {105: 4, 194: 1, 93: 4, 297: nan, 131: 5}, 'pcr_a_p1_r1': {105: 4.0, 194: 5.0, 93: 2.0, 297: 2.0, 131: 3.0}, 'pcr_a_p1_r2': {105: 3.0, 194: 4.0, 93: 4.0, 297: 3.0, 131: 3.0}, 'pcr_a_p1_r3': {105: 4.0, 194: 5.0, 93: 2.0, 297: 3.0, 131: 3.0}, 'pcr_a_p1_r4': {105: 4, 194: 5, 93: 3, 297: 2, 131: 4}, 'pcr_a_p1_r5': {105: 4.0, 194: 5.0, 93: 2.0, 297: 4.0, 131: 5.0}, 'pcr_a_p1_r6': {105: 4, 194: 5, 93: 2, 297: 2, 131: 2}, 'pcr_a_p1_r7': {105: 4, 194: 4, 93: 4, 297: 3, 131: 4}, 'pcr_a_p1_r8': {105: 4, 194: 5, 93: 3, 297: 4, 131: 4}, 'pcr_a_p1_r9': {105: 4, 194: 4, 93: 4, 297: 4, 131: 5}, 'pcr_a_p1_r10': {105: 4, 194: 5, 93: 4, 297: 2, 131: 4}, 'pcr_a_p1_r11': {105: 4.0, 194: 5.0, 93: 4.0, 297: 2.0, 131: 4.0}, 'pcr_a_p1_r12': {105: 4, 194: 5, 93: 3, 297: 2, 131: 3}, 'pcr_a_p1_r13': {105: 4, 194: 5, 93: 2, 297: 2, 131: 3}, 'pcr_a_p1_r14': {105: 4.0, 194: 4.0, 93: 2.0, 297: 2.0, 131: 3.0}, 'pcr_a_p1_r15': {105: 4.0, 194: 5.0, 93: 3.0, 297: 2.0, 131: 5.0}, 'pcr_a_p2_r1': {105: 5.0, 194: 5.0, 93: 4.0, 297: 2.0, 131: 5.0}, 'pcr_a_p2_r2': {105: 4.0, 194: 5.0, 93: 3.0, 297: 2.0, 131: 5.0}, 'pcr_a_p2_r3': {105: 3.0, 194: 5.0, 93: 4.0, 297: 4.0, 131: 4.0}, 'pcr_a_p2_r4': {105: 4, 194: 4, 93: 2, 297: 3, 131: 5}, 'pcr_a_p2_r5': {105: 5.0, 194: 5.0, 93: 5.0, 297: 3.0, 131: 4.0}, 'pcr_a_p2_r6': {105: 3.0, 194: 1.0, 93: 1.0, 297: 1.0, 131: 2.0}, 'pcr_a_p2_r7': {105: 5, 194: 4, 93: 3, 297: 3, 131: 5}, 'pcr_a_p2_r8': {105: 5, 194: 3, 93: 3, 297: 2, 131: 4}, 'pcr_a_p2_r9': {105: 5, 194: 2, 93: 2, 297: 4, 131: 3}, 'pcr_a_p2_r10': {105: 3, 194: 2, 93: 1, 297: 2, 131: 2}, 'pcr_a_p2_r11': {105: 4, 194: 3, 93: 2, 297: 3, 131: 4}, 'pcr_a_p2_r12': {105: 5.0, 194: 4.0, 93: 3.0, 297: 3.0, 131: 5.0}, 'pcr_a_p2_r13': {105: 4, 194: 5, 93: 1, 297: 1, 131: 5}, 'pcr_a_p2_r14': {105: 5.0, 194: 5.0, 93: 3.0, 297: 2.0, 131: 4.0}, 'pcr_a_p2_r15': {105: 2.0, 194: 5.0, 93: 5.0, 297: 2.0, 131: 2.0}, 'det_a_p1_r1': {105: 3.0, 194: 4.0, 93: 4.0, 297: 4.0, 131: 5.0}, 'det_a_p1_r2': {105: 4.0, 194: 4.0, 93: 4.0, 297: 3.0, 131: 4.0}, 'det_a_p1_r3': {105: 4.0, 194: 4.0, 93: 4.0, 297: 3.0, 131: 5.0}, 'det_a_p1_r4': {105: 4.0, 194: 4.0, 93: 4.0, 297: 2.0, 131: 4.0}, 'det_a_p1_r5': {105: 3.0, 194: 1.0, 93: 2.0, 297: 4.0, 131: 1.0}, 'det_a_p1_r6': {105: 3, 194: 5, 93: 4, 297: 2, 131: 5}, 'det_a_p1_r7': {105: 4.0, 194: 5.0, 93: 4.0, 297: 2.0, 131: 5.0}, 'det_a_p1_r8': {105: 3, 194: 5, 93: 4, 297: 2, 131: 5}, 'det_a_p1_r9': {105: 3, 194: 1, 93: 2, 297: 3, 131: 1}, 'det_a_p1_r10': {105: 4.0, 194: 1.0, 93: 2.0, 297: 2.0, 131: 1.0}, 'det_a_p1_r11': {105: 4, 194: 4, 93: 3, 297: 2, 131: 5}, 'det_a_p1_r12': {105: 2, 194: 5, 93: 4, 297: 3, 131: 5}, 'det_a_p1_r13': {105: 4.0, 194: 5.0, 93: 4.0, 297: 4.0, 131: 5.0}, 'det_a_p1_r14': {105: 2, 194: 1, 93: 2, 297: 3, 131: 5}, 'det_a_p1_r15': {105: 1.0, 194: 2.0, 93: 3.0, 297: 5.0, 131: 1.0}, 'det_a_p2_r1': {105: 3.0, 194: 5.0, 93: 4.0, 297: 3.0, 131: 3.0}, 'det_a_p2_r2': {105: 2.0, 194: 5.0, 93: 3.0, 297: 4.0, 131: 5.0}, 'det_a_p2_r3': {105: 3.0, 194: 4.0, 93: 2.0, 297: 2.0, 131: 4.0}, 'det_a_p2_r4': {105: 4, 194: 4, 93: 4, 297: 2, 131: 3}, 'det_a_p2_r5': {105: 5, 194: 3, 93: 5, 297: 4, 131: 2}, 'det_a_p2_r6': {105: 3, 194: 4, 93: 3, 297: 3, 131: 4}, 'det_a_p2_r7': {105: 3.0, 194: 2.0, 93: 3.0, 297: 3.0, 131: 1.0}, 'det_a_p2_r8': {105: 3, 194: 4, 93: 4, 297: 2, 131: 5}, 'det_a_p2_r9': {105: 3, 194: 3, 93: 3, 297: 2, 131: 5}, 'det_a_p2_r10': {105: 4, 194: 4, 93: 4, 297: 4, 131: 5}, 'det_a_p2_r11': {105: 3, 194: 2, 93: 3, 297: 2, 131: 5}, 'det_a_p2_r12': {105: 2, 194: 4, 93: 2, 297: 3, 131: 5}, 'det_a_p2_r13': {105: 4, 194: 4, 93: 4, 297: 4, 131: 4}, 'det_a_p2_r14': {105: 4, 194: 4, 93: 4, 297: 4, 131: 4}, 'det_a_p2_r15': {105: 4, 194: 5, 93: 3, 297: 2, 131: 5}, 'lid_a_p1_r1': {105: 3.0, 194: 4.0, 93: 4.0, 297: nan, 131: 4.0}, 'lid_a_p1_r2': {105: 3.0, 194: 4.0, 93: 3.0, 297: nan, 131: 3.0}, 'lid_a_p1_r3': {105: 3, 194: 4, 93: 3, 297: nan, 131: 3}, 'lid_a_p1_r4': {105: 3, 194: 5, 93: 4, 297: nan, 131: 4}, 'lid_a_p1_r5': {105: 4.0, 194: 4.0, 93: 2.0, 297: nan, 131: 4.0}, 'lid_a_p1_r6': {105: 4.0, 194: 5.0, 93: 4.0, 297: nan, 131: 4.0}, 'lid_a_p1_r7': {105: 3.0, 194: 4.0, 93: 4.0, 297: nan, 131: 4.0}, 'lid_a_p1_r8': {105: 1.0, 194: 5.0, 93: 5.0, 297: nan, 131: 5.0}, 'lid_a_p1_r9': {105: -1.5, 194: 5, 93: 5, 297: nan, 131: 3}, 'lid_a_p1_r10': {105: 1, 194: 3, 93: 3, 297: nan, 131: 5}, 'lid_a_p1_r11': {105: 2, 194: 2, 93: 3, 297: nan, 131: 2}, 'lid_a_p1_r12': {105: 1, 194: 1, 93: 3, 297: nan, 131: 1}, 'lid_a_p1_r13': {105: 3, 194: 5, 93: 4, 297: nan, 131: 5}, 'lid_a_p1_r14': {105: 3.0, 194: 4.0, 93: 4.0, 297: nan, 131: 3.0}, 'lid_a_p1_r15': {105: 3, 194: 3, 93: 3, 297: nan, 131: 3}, 'lid_a_p2_r1': {105: 3, 194: 4, 93: 3, 297: 4, 131: 5}, 'lid_a_p2_r2': {105: 3, 194: 4, 93: 3, 297: 3, 131: 4}, 'lid_a_p2_r3': {105: 4, 194: 4, 93: 4, 297: 3, 131: 3}, 'lid_a_p2_r4': {105: 3, 194: 5, 93: 2, 297: 2, 131: 4}, 'lid_a_p2_r5': {105: 5, 194: 5, 93: 3, 297: 2, 131: 3}, 'lid_a_p2_r6': {105: 4, 194: 4, 93: 3, 297: 4, 131: 4}, 'lid_a_p2_r7': {105: 5, 194: 5, 93: 5, 297: 4, 131: 4}, 'lid_a_p2_r8': {105: 2, 194: 3, 93: 3, 297: 2, 131: 5}, 'lid_a_p2_r9': {105: 3, 194: 4, 93: 3, 297: 2, 131: 4}, 'lid_a_p2_r10': {105: 3.0, 194: 5.0, 93: 3.0, 297: 4.0, 131: 5.0}, 'lid_a_p2_r11': {105: 3, 194: 3, 93: 3, 297: 3, 131: 3}, 'lid_a_p2_r12': {105: 2.0, 194: 4.0, 93: 2.0, 297: 2.0, 131: 5.0}, 'lid_a_p2_r13': {105: 5.0, 194: 3.0, 93: 4.0, 297: 3.0, 131: 4.0}, 'lid_a_p2_r14': {105: 3.0, 194: 5.0, 93: 4.0, 297: 2.0, 131: 3.0}, 'lid_a_p2_r15': {105: 4.0, 194: 5.0, 93: 3.0, 297: 2.0, 131: 4.0}}
)

df.loc[:,df.columns.str.contains("_a_")] = df.apply(pd.to_numeric, errors='coerce') 
df.loc[:,df.columns.str.contains("_a_")] = df.mask((df < 1) | (df > 5))
df

CodePudding user response:

I think you df.loc works as you expect it to work but you apply the mask on the full df. I would do this instead:

df_a_columns = df.loc[:,df.columns.str.contains("_a_")] 
df.loc[:,df.columns.str.contains("_a_")] = df_a_columns.mask((df_a_columns < 1) | (df_a_columns > 5))
  • Related