I want to create data frame based on some input condition -but seems unable to use dataframe out of-CodePudding

I want to create data frame based on some input condition -but seems unable to use dataframe out of if else logic

    if(table_1_src =="file") {        
      val df1= spark.read.format("csv")
     .option("path", table_2_path)
     .option("header",true)
     .option("inferSchema",true)
     .load
     }
   df1.show()
    Error: Value not found df1while calling df1.show()

CodePudding user response：

You are declaring the data frame in an inner scope, so it is not visible in the place you want to call it.

You can only reference variables that were declared on at least the same 'level' as the usage place. e.g.

if(table_1_src =="file") {        
  val df1 = spark.read.format("csv")
    .option("path", table_2_path)
    .option("header",true)
    .option("inferSchema",true)
    .load
  df1.show()
}

CodePudding user response：

you are defining variable inside the scope of if condition and that's why it's not available.

This should work.

val df1 = if(table_1_src =="file") {        
  val df1= spark.read.format("csv")
 .option("path", table_2_path)
 .option("header",true)
 .option("inferSchema",true)
 .load
 df1
 }
df1.show()