Is there anything in Pandas similar to dplyr's 'list columns'-CodePudding

I'm currently transitioning from R to Python in my data analysis, and there's one thing I haven't seen in any tutorials out there: is there anything in Pandas similar to dplyr's 'list columns' ?

Link to refence: https://www.rstudio.com/resources/webinars/how-to-work-with-list-columns/

CodePudding user response：

pandas will accept any object type, including lists, in an object type column.

df = pd.DataFrame()
df['genre']=['drama, comedy, action', 'romance, sci-fi, drama','horror']
df.genre = df.genre.str.split(', ')
print(df, '\n', df.genre.dtype, '\n', type(df.genre[0]))

# Output:

                      genre
0   [drama, comedy, action]
1  [romance, sci-fi, drama]
2                  [horror]
 object
 <class 'list'>

We can see that:

genre is a column of lists.
The dtype of the genre column is object
The type of the first value of genre is list.

There are a number of str functions that work with lists.

For example:

print(df.genre.str.join(' | '))

# Output:

0     drama | comedy | action
1    romance | sci-fi | drama
2                      horror
Name: genre, dtype: object

print(df.genre.str[::2])

# Output:

0     [drama, action]
1    [romance, drama]
2            [horror]
Name: genre, dtype: object

Others can typically be done with an apply function if there isn't a built-in method:

print(df.genre.apply(lambda x: max(x)))

# Output:

0     drama
1    sci-fi
2    horror
Name: genre, dtype: object

See the documentation for more... pandas str functions

As for nesting dataframes within one another, it is possible but, I believe it's considered an anti-pattern, and pandas will fight you the whole way there:

data = {'df1': df, 'df2': df}
df2 = pd.Series(data.values(), data.keys()).to_frame()
df2.columns = ['dfs']
print(df2)

# Output:

                                                   dfs
df1                        genre
0   [drama, comedy...
df2                        genre
0   [drama, comedy...

print(df2['dfs'][0])

# Output:

                      genre
0   [drama, comedy, action]
1  [romance, sci-fi, drama]
2                  [horror]

See:

Link1
Link2

A possibly acceptable work around, would be storing them as numpy arrays:

df2 = df2.applymap(np.array)
print(df2)
print(df2['dfs'][0])

# Output:

                                                   dfs
df1  [[[drama, comedy, action]], [[romance, sci-fi,...
df2  [[[drama, comedy, action]], [[romance, sci-fi,...

array([[list(['drama', 'comedy', 'action'])],
       [list(['romance', 'sci-fi', 'drama'])],
       [list(['horror'])]], dtype=object)