Home > other >  Spark connection hive 3. 3.0 x database, the query is less than the data in the table
Spark connection hive 3. 3.0 x database, the query is less than the data in the table

Time:09-24

The Exception in the thread "main" Java. Lang. NoSuchMethodError: org.. Apache hadoop. Hive. Ql. Exec. Utilities. CopyTableJobPropertiesToConf (Lorg/apache/hadoop/hive ql/plan/TableDesc; Lorg/apache/hadoop/conf/Configuration) V
The at org. Apache. Spark. SQL. Hive. HadoopTableReader $. InitializeLocalJobConfFunc (TableReader. Scala: 382)
The at org. Apache. Spark. SQL. Hive. HadoopTableReader. $$createOldHadoopRDD anonfun $1 (308). TableReader scala:
At org. Apache. Spark. SQL. Hive. HadoopTableReader $$$1 $adapted createOldHadoopRDD anonfun (TableReader. Scala: 308)
The at org. Apache. Spark. RDD. HadoopRDD. $$getJobConf anonfun $8 (181). HadoopRDD scala:
The at org. Apache. Spark. RDD. HadoopRDD. $$getJobConf anonfun $8 $adapted (HadoopRDD. Scala: 181)
At the scala. Option. Foreach (407) Option. The scala:
At org. Apache. Spark. RDD. HadoopRDD. $$getJobConf anonfun $6 (181). HadoopRDD scala:
At the scala. Option. GetOrElse (189) Option. The scala:
The at org. Apache. Spark. RDD. HadoopRDD. GetJobConf (HadoopRDD. Scala: 178)
The at org. Apache. Spark. RDD. HadoopRDD. GetPartitions (HadoopRDD. Scala: 201)
The at org. Apache. Spark. RDD. RDD. $$partitions anonfun $2 (RDD. Scala: 276)
At the scala. Option. GetOrElse (189) Option. The scala:
The at org. Apache. Spark. RDD. RDD. Partitions (RDD. Scala: 272)
The at org. Apache. Spark. RDD. MapPartitionsRDD. GetPartitions (49) MapPartitionsRDD. Scala:
The at org. Apache. Spark. RDD. RDD. $$partitions anonfun $2 (RDD. Scala: 276)
At the scala. Option. GetOrElse (189) Option. The scala:
The at org. Apache. Spark. RDD. RDD. Partitions (RDD. Scala: 272)
The at org. Apache. Spark. RDD. MapPartitionsRDD. GetPartitions (49) MapPartitionsRDD. Scala:
The at org. Apache. Spark. RDD. RDD. $$partitions anonfun $2 (RDD. Scala: 276)
At the scala. Option. GetOrElse (189) Option. The scala:
The at org. Apache. Spark. RDD. RDD. Partitions (RDD. Scala: 272)
The at org. Apache. Spark. RDD. MapPartitionsRDD. GetPartitions (49) MapPartitionsRDD. Scala:
The at org. Apache. Spark. RDD. RDD. $$partitions anonfun $2 (RDD. Scala: 276)
At the scala. Option. GetOrElse (189) Option. The scala:
The at org. Apache. Spark. RDD. RDD. Partitions (RDD. Scala: 272)
The at org. Apache. Spark. RDD. MapPartitionsRDD. GetPartitions (49) MapPartitionsRDD. Scala:
The at org. Apache. Spark. RDD. RDD. $$partitions anonfun $2 (RDD. Scala: 276)
At the scala. Option. GetOrElse (189) Option. The scala:
The at org. Apache. Spark. RDD. RDD. Partitions (RDD. Scala: 272)
The at org. Apache. Spark. RDD. MapPartitionsRDD. GetPartitions (49) MapPartitionsRDD. Scala:
The at org. Apache. Spark. RDD. RDD. $$partitions anonfun $2 (RDD. Scala: 276)
At the scala. Option. GetOrElse (189) Option. The scala:
The at org. Apache. Spark. RDD. RDD. Partitions (RDD. Scala: 272)
The at org. Apache. Spark. SQL. Execution. SparkPlan. ExecuteTake (SparkPlan. Scala: 408)
The at org. Apache. Spark. SQL. Execution. CollectLimitExec. ExecuteCollect (limit. Scala: 47)
At org. Apache. Spark. SQL. The Dataset. CollectFromPlan (3482) Dataset. Scala:
At org. Apache. Spark. SQL. The Dataset. $$head anonfun $1 (2581) Dataset. Scala:
The at org. Apache. Spark. SQL. The Dataset. $$withAction anonfun $1 (3472) Dataset. Scala:
At org. Apache. Spark. SQL. Execution. SQLExecution $$anonfun $withNewExecutionId $4 (100). SQLExecution scala:
The at org. Apache. Spark. SQL. Execution. SQLExecution $. WithSQLConfPropagated (SQLExecution. Scala: 160)
The at org. Apache. Spark. SQL. Execution. SQLExecution $. WithNewExecutionId (SQLExecution. Scala: 87)
At org. Apache. Spark. SQL. The Dataset. WithAction (3468) Dataset. Scala:
The at org. Apache. Spark. SQL. The Dataset. The head (2581) Dataset. Scala:
The at org. Apache. Spark. SQL. The Dataset. Take (2788) Dataset. Scala:
At org. Apache. Spark. SQL. The Dataset. GetRows (297) Dataset. Scala:
At org. Apache. Spark. SQL. The Dataset. ShowString (334) Dataset. Scala:
At org. Apache. Spark. SQL. The Dataset. The show (816) Dataset. Scala:
At org. Apache. Spark. SQL. The Dataset. The show (775) Dataset. Scala:
The at test. SparkHive $. The main (SparkHive scala: 18)
The at test. SparkHive. Main (SparkHive. Scala)

We can create the tables in the spark, can query to create a table of data in the spark,

CodePudding user response:

Look at this error, is not yet find the reasons why copyTableJobPropertiesToConf table, you lack of spark - hive package reference?
  • Related