Home > Software engineering >  Unable to Writing data in Delta Format
Unable to Writing data in Delta Format

Time:09-14

I have upload the parquet data format file into databricks file System. Now I want to to store this parquet file data into Delta Table (Delta Format). But unable to do. My code is:

spark.read.option("inferschema",true).option("header",true).csv("/FileStore/tables/ExecutiveSummary.txt").
write.format("delta").mode("overwrite").save("/FileStore/tables/delta_train/")

It show me syntax error. What is the issue in this ?

I am using databricks community edition

CodePudding user response:

You can't simply split code into two lines. Your first line is finishing with . character - it's incorrect in python. You need to use either this variant (with \ as continuation marker, and . put to the write word):

spark.read.option("inferschema",true).option("header",true).csv("/FileStore/tables/ExecutiveSummary.txt") \
  .write.format("delta").mode("overwrite").save("/FileStore/tables/delta_train/")

or this variant, with () around your expression, but still you need to put . in correct place:

(spark.read.option("inferschema",true).option("header",true).csv("/FileStore/tables/ExecutiveSummary.txt")
  .write.format("delta").mode("overwrite").save("/FileStore/tables/delta_train/"))
  • Related