Home > other >  Spark save data to mysql, StringType type problems
Spark save data to mysql, StringType type problems

Time:09-21

Spark saved to a data type to mysql, if using append mode, completely according to your definition of data table structure insert data, there is no problem;

But if you use the override mode, the spark will automatically build table, type String corresponds to the Text type, what method can solve? I need to corresponding with the varchar type,

Tried some custom database dialect, but because of the spark has defined the mysql dialect, after I register their own dialect, the results will match the two (my custom and spark's own), and spark to own is to use private modifiers, I also can't modify,

Please advise what method?
The spark version 2.1.0
  • Related