What is the best way to iterate over Pandas rows in my case?-CodePudding

I have a complexity problem with iterations over Pandas Rows. I have a dataset with over 30k rows and i need for each one, add a new column with values from specifics columns.

belongs_node_df = pd.DataFrame.from_records(belongs_node, columns=['hashtag', 'tweets_id', 'tokenized_text','sentiment_compound'])
posted_node_df = pd.DataFrame.from_records(posted_node, columns=['username', 'num_followers', 'tweets_id'])
df_user_hashtag = pd.merge(posted_node_df, belongs_node_df, on='tweets_id', how='outer').sort_values('username')
df_user_hashtag['p'] = None

for i in range(len(df_user_hashtag)):
    df_user_hashtag['p'][i] = 3 * df_user_hashtag['num_followers'][i]\df_user_hashtag['sentiment_compound'][i]

There is an efficient way to make this operation for each row? Thanks a lot. :)

CodePudding user response：

you should not iterate over the rows ... this breaks pretty much all the benefits you get from using pandas

df_user_hashtag['p'] = 3 * df_user_hashtag['num_followers'] / df_user_hashtag['sentiment_compound']

CodePudding user response：

import pandas as pd

df = pd.read_excel('/content/Endereços CS 2021.xlsx')

data = df.values.tolist()

for d in data:
    print(d)

In this example i read my excel file and than convert it to a list by using the metods .values.tolist() then i did what i said earlier, i iterate over each row in my list.

The output was that:

['Ana Lara', 'Alfa', 'Amigo']
['Ana Suely', 'Alfa', 'Amigo']
['Izabelly', 'Alfa', 'Amigo']
['Carol Loiola', 'Alfa', 'Amigo']
['Yasmin', 'Alfa', 'Amigo']
['Mariana', 'Alfa', 'Amigo']
['Tereza', 'Alfa', 'Amigo']
['Rívia', 'Alfa', 'Amigo']
['Stefany', 'Alfa', 'Amigo']
['Maria Eduarda', 'Alfa', 'Amigo']
['Meyssa', 'Alfa', 'Amigo']
['Arthur Figueiró', 'Epsilon', 'Amigo']
['Andriw', 'Epsilon', 'Amigo']
['Gabriel', 'Epsilon', 'Amigo']
['Tiago ', 'Epsilon', 'Amigo']
['João Pedro', 'Epsilon', 'Amigo']
['Carlos', 'Epsilon', 'Amigo']
['José Neto', 'Epsilon', 'Amigo']
['Raissa ', 'Beta', 'Pesquisador']
['Lara Yasmin', 'Beta', 'Pesquisador']
['Letícia', 'Beta', 'Pesquisador']
['Thalita Melo', 'Beta', 'Pesquisador']
['Isabel', 'Beta', 'Pesquisador']
['Melyssa', 'Beta', 'Excursionista']
['Sarah Gabrielle', 'Beta', 'Excursionista']
['Daniel Fernandes', 'Delta', 'Pioneiro']
['Arthur Soares', 'Delta', 'Pioneiro']
['Guido', 'Delta', 'Pioneiro']
['Emanoel', 'Delta', 'Pioneiro']
['Flávio', 'Delta', 'Pioneiro']
['Iohannes', 'Delta', 'Pioneiro']
['Lucas', 'Delta', 'Pesquisador']
['Beatriz Gillianne (Bia)', 'Sigma', 'Guia']
['Emilly Vitória', 'Sigma', 'Excursionista']
['Adriana', 'Sigma', 'Guia']
['Jade', 'Sigma', 'Guia']
['Sarah Leocádio', 'Sigma', 'Guia']
['Maria Eduarda', 'Sigma', 'Guia']

CodePudding user response：

I solved this using @Riley suggestion from the comments.

First, I made a function which gets the value I want:

def get_p(tfidf, num_followers, compund):
    return (tfidf * num_followers) * compund

Second, I used Numpy's vectorize function to call my function using vectorization:

vfunc = numpy.vectorize(get_p)
df_user_hashtag['p'] = vfunc(1, df_user_hashtag['num_followers'], df_user_hashtag['sentiment_compound'])

That's all!

CodePudding user response：

you can use the methode .tolist() and iterate trough each row of the list