Pandas if any value is greater than x subtract x with multiple dataframes different column names-CodePudding

I am processing 20 or so data frames each with different column names and in each dataframe, if there is a numeric value greater than x, I want to subtract x.

for example, in df one

  Name             ID           DOB
Joe Smith    10000000021021   01011901
Jane Smith   10000001074600   07025901

but then in df two I may have

Group            Office         OrderNum
20151        10000000060021     20051568
70386            90021          20051568

and these column names are different in each df but if a numeric value is greater than 10000000000000, then subtract that amount

CodePudding user response：

You could use applymap:

THRESH = 10000000000000
df.applymap(lambda x: x-THRESH if isinstance(x, (int, float)) and x>THRESH else x)

example output:

       Name       ID      DOB
Joe   Smith    21021  1011901
Jane  Smith  1074600  7025901

CodePudding user response：

You can use select_dtypes and work on those columns:

num = 1e13
dfs = [df1, df2]

for i in dfs:
    cols = i.select_dtypes("int64").columns # or use 'number'
    i[cols] = i[cols].mask(i[cols]>num, i[cols]-num)
    print (i)

         Name       ID      DOB
0   Joe Smith    21021  1011901
1  Jane Smith  1074600  7025901

   Group  Office  OrderNum
0  20151   60021  20051568
1  70386   90021  20051568