I have a text file like below, I wanna create a data frame, and when I read this file using pyspark I get only three columns of the data frame like below (second image). could someone help me to create a data frame with different number columns in each row

123, 45, 56
123, 45 ,56, 2022/06/15, 56.78
123, 45, 56 , 2022/06/15, 56, 566, 7890
123, 45, 56 , , 56, , 7890
123, ,56,78

CodePudding user response：

This is not possible in Spark or pandas, because they are two-dimensional dataframes with a fixed number of rows and columns. So, you must have an equal number of rows for each column and also an equal number of columns for each row.