Home > other >  java.io.InvalidClassException: org.apache.tinkerpop.gremlin.hadoop.structure.HadoopConfiguration; un
java.io.InvalidClassException: org.apache.tinkerpop.gremlin.hadoop.structure.HadoopConfiguration; un

Time：12-09

I am totally new to Spark, My spark master is on GCP Kubernetes Standard cluster. I am using a gremlin client to trigger the spark job, on my local machine configuration it works totally fine and the job gets completed, But on Kubernetes, I am seeing the below error.

Gremlin Query and error

==>tinkerpop.hadoop activated
gremlin> :plugin use tinkerpop.spark
==>tinkerpop.spark activated
gremlin> graph = GraphFactory.open('/opt/janusgraph-0.6.2/conf/hadoop-graph/read-hbase-standalone-cluster-stg-bigtable-spark.properties')
==>hadoopgraph[hbaseinputformat->nulloutputformat]
gremlin> 
gremlin> g = graph.traversal().withComputer(SparkGraphComputer)
==>graphtraversalsource[hadoopgraph[hbaseinputformat->nulloutputformat], sparkgraphcomputer]
gremlin> g.V().count()
14:01:48 WARN  org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer  - class org.apache.hadoop.mapreduce.lib.output.NullOutputFormat does not implement PersistResultGraphAware and thus, persistence options are unknown -- assuming all options are possible
14:02:03 WARN  org.apache.spark.scheduler.TaskSetManager  - Lost task 6.0 in stage 0.0 (TID 6, XX.XX.XX.XX, executor 0): java.io.InvalidClassException: org.apache.tinkerpop.gremlin.hadoop.structure.HadoopConfiguration; unable to create instance
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2198)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.readArray(ObjectInputStream.java:2117)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1658)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.readArray(ObjectInputStream.java:2117)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1658)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.readArray(ObjectInputStream.java:2117)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1658)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:488)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:446)
        at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:488)
        at jdk.internal.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1046)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2340)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:488)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:446)
        at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:488)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1046)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2340)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:488)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:446)
        at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:76)
        at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:115)
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:85)
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52)
        at org.apache.spark.scheduler.Task.run(Task.scala:127)
        at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:446)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1377)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:449)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.reflect.InvocationTargetException
        at jdk.internal.reflect.GeneratedSerializationConstructorAccessor270.newInstance(Unknown Source)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
        at java.base/java.io.ObjectStreamClass.newInstance(ObjectStreamClass.java:963)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2195)
        ... 77 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.commons.configuration2.interpol.ConfigurationInterpolator
        at org.apache.commons.configuration2.AbstractConfiguration.installDefaultInterpolator(AbstractConfiguration.java:455)
        at org.apache.commons.configuration2.AbstractConfiguration.<init>(AbstractConfiguration.java:132)
        ... 81 more

14:02:03 ERROR org.apache.spark.scheduler.TaskSetManager  - Task 8 in stage 0.0 failed 4 times; aborting job
org.apache.spark.SparkException: Job aborted due to stage failure: Task 8 in stage 0.0 failed 4 times, most recent failure: Lost task 8.3 in stage 0.0 (TID 32, 10.12.11.7, executor 1): java.io.InvalidClassException: org.apache.tinkerpop.gremlin.hadoop.structure.HadoopConfiguration; unable to create instance
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2198)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.readArray(ObjectInputStream.java:2117)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1658)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.readArray(ObjectInputStream.java:2117)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1658)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.readArray(ObjectInputStream.java:2117)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1658)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:488)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:446)
        at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:488)
        at jdk.internal.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1046)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2340)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:488)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:446)
        at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:488)
        at jdk.internal.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1046)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2340)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:488)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:446)
        at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:76)
        at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:115)
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:85)
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52)
        at org.apache.spark.scheduler.Task.run(Task.scala:127)
        at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:446)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1377)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:449)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.reflect.InvocationTargetException
        at jdk.internal.reflect.GeneratedSerializationConstructorAccessor238.newInstance(Unknown Source)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
        at java.base/java.io.ObjectStreamClass.newInstance(ObjectStreamClass.java:963)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2195)
        ... 76 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.commons.configuration2.interpol.ConfigurationInterpolator
        at org.apache.commons.configuration2.AbstractConfiguration.installDefaultInterpolator(AbstractConfiguration.java:455)
        at org.apache.commons.configuration2.AbstractConfiguration.<init>(AbstractConfiguration.java:132)
        ... 80 more

Config File--> read-hbase-standalone-cluster-stg-bigtable-spark.properties

#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#
# Hadoop Graph Configuration
#
gremlin.graph=org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph
gremlin.hadoop.graphReader=org.janusgraph.hadoop.formats.hbase.HBaseInputFormat
gremlin.hadoop.graphWriter=org.apache.hadoop.mapreduce.lib.output.NullOutputFormat

gremlin.hadoop.jarsInDistributedCache=true
gremlin.hadoop.inputLocation=none
gremlin.hadoop.outputLocation=output
gremlin.spark.persistContext=true

#
# JanusGraph HBase InputFormat configuration
#
#janusgraphmr.ioformat.conf.storage.backend=hbase
#janusgraphmr.ioformat.conf.storage.hostname=localhost
#janusgraphmr.ioformat.conf.storage.port=8586
#janusgraphmr.ioformat.conf.storage.hbase.table=janusgraph
janusgraphmr.ioformat.conf.storage.backend=hbase
janusgraphmr.ioformat.conf.storage.hbase.ext.hbase.client.connection.impl=com.google.cloud.bigtable.hbase2_x.BigtableConnection
janusgraphmr.ioformat.conf.storage.hbase.ext.google.bigtable.project.id=**********
janusgraphmr.ioformat.conf.storage.hbase.ext.google.bigtable.instance.id=*********
janusgraphmr.ioformat.conf.storage.hbase.table=**********
janusgraphmr.ioformat.conf.storage.hbase.ext.hbase.regionsizecalculator.enable=false

# This defines the indexing backend configuration used while writing data to JanusGraph.
janusgraphmr.ioformat.conf.index.search.backend=elasticsearch
janusgraphmr.ioformat.conf.index.search.hostname=**********

#
# SparkGraphComputer Configuration
#
spark.master=spark://spark-master:7077
spark.executor.memory=1g
spark.executor.extraClassPath=/opt/janusgraph-0.6.2/lib/*
spark.serializer=org.apache.spark.serializer.KryoSerializer
spark.kryo.registrator=org.janusgraph.hadoop.serialize.JanusGraphKryoRegistrator```

CodePudding user response：

With reference to this https://li-boxuan.medium.com/configure-janusgraph-0-6-0-for-spark-64bd7628fb77 blog. spark.executor.extraClassPath=/opt/janusgraph-0.6.2/lib/* mentioned in the gremlin reference config files, these libraries should be present in both Master and Worker nodes.

Page link：https//www.codepudding.com/other/636935.html

Prev:How to set --num-executors in dataproc cluster
Next:Writing to a JSON column type in BigQuery using Spark
  • Related

About Us:  Contact Us      Terms of Service       Privacy Policy

Copyright © 2010-2023，Powered By CodePudding