Home > other >  Consult a CSV SparkSQL analytic embedded with json
Consult a CSV SparkSQL analytic embedded with json

Time:09-25

Open in excel, paste out data is like this:
Uid, uuid, the event_type, event_data created_at
- the action, 123456, abcdefabcdefabcdef, "{" "duration "" :" "20" ", "" questionId" ":" "123456" "} ", 1476201605

The event_data here is a json, use the following code reads:
Val usereventDF=spark. Read. The format (" CSV "). The option (" header ", "true"). The option (" inferSchema ", "true")
The load ("/Users/XXX/Desktop/event. CSV ")
. CreateOrReplaceTempView (" t_event ")

When select * will find created_at field output is according to the front of json comma identified:
+ -- -- -- -- -- -- - + -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - + -- -- -- -- -- -- -- -- -- -- -- -- -- - + -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - + -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- +
Uid | | uuid | the event_type | event_data | created_at |
+ -- -- -- -- -- -- - + -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - + -- -- -- -- -- -- -- -- -- -- -- -- -- - + -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - + -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- +
123456 | | abcdefabcdefabcdef... | some - action | "{" "duration "" :"... "|" "questionId" :... |

Even with get_json_object, also find direct output '$' is a "{"

Data is given by a third party for the parse the CSV table set of json in correct position, thank you!
  • Related