If you append 2 pandas dataframes and assign to a variable, will it take space in memory?

For instance, if I have df_1 which weights 500MB and df2 also weighting 500MB,

after running this code below:

df_append = df_1.append(df_2, ignore_index = True)

Will my memory usage be 2000MB (500 500 1000), or will it be 1000MB?

Is this all about method we're using, concat, append and merge?

CodePudding user response：

If df_1 and df2 both use 500MB of memory, then df_append = df_1.append(df_2, ignore_index = True) will build a new dataframe which will use more or less 1000 new MB, for a 2000MB total.

But beware, if a dataframe can be serialized into a 500MB file (for example in cdv format), it is likely to use far more memory...

CodePudding user response：

For example, if you assign a var with the use of 1MB memory and the second var with the use of 1MB if you combine both ones and form a new var, then the new var take 2MB memory. Same as here. It will take approximately 1000MB, approximately huh... Don't think exactly. Because your df_1 could be used 500.000245(not exactly 500) and df_2 could be used 500.000something...(not exactly 500).

I hope you understand.