How to get StructType object out of a StructType in spark java?-CodePudding

I'm working on this spark java application and I wanted to access structtype object in a structtype object. For instance-

When we take schema of spark dataframe it looks something like this-

root
 |-- name: struct (nullable = true)
 |    |-- firstname: string (nullable = true)
 |    |-- middlename: string (nullable = true)
 |    |-- lastname: string (nullable = true)
 |-- language: string (nullable = true)
 |-- fee: integer (nullable = true)

I wanted to grab the name as a structtype so that I can analyze it further. It would make a sort of chain. But the problem is that at the root level or any level, we can only extract structfield out of structtype and not other structtype.

StructType st = df.schema(); --> we get root level structtype
st.fields(); --> give us array of structfields but if I take name as a structfield i will lose all the fields inside it as 'name' is a StructType and I want to have it as it is.
StructType name = out of st  --> this is what I want to achieve.

CodePudding user response：

You can use the parameters and methods mentioned in the official documentation:

schema = StructType([StructField('name', StructType([StructField('firstname', StringType()), StructField('middlename', StringType()), StructField('lastname', StringType())])), StructField('language', StringType()), StructField('fee', IntegerType())])

for f in schema.fields:
  if (f.name == "name"):
    print(f.dataType)
    for f2 in f.dataType.fields:
      print(f2.name)

[Out]:
StructType([StructField('firstname', StringType(), True), StructField('middlename', StringType(), True), StructField('lastname', StringType(), True)])

firstname
middlename
lastname