I'm just getting started with Pig and I'm facing lots of issues with running my first program. Any help is much appreciated.

I've tried resolving using these:

ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. org/apache/hadoop/hbase/filter/WritableByteArrayComparable

and

Pig Installation error: ERROR pig.Main: ERROR 2998: Unhandled internal error

but none of them seem to work. Can someone give a more detailed solution of what needs to be done.

Pig version: 0.17.0

Stack Trace:

Pig Stack Trace
---------------
ERROR 2998: Unhandled internal error. org/python/google/common/collect/Lists

java.lang.NoClassDefFoundError: org/python/google/common/collect/Lists
    at org.apache.pig.tools.pigstats.mapreduce.MRJobStats.getTaskReports(MRJobStats.java:533)
    at org.apache.pig.tools.pigstats.mapreduce.MRJobStats.addMapReduceStatistics(MRJobStats.java:355)
    at org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil.addSuccessJobStats(MRPigStatsUtil.java:232)
    at org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil.accumulateStats(MRPigStatsUtil.java:164)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:379)
    at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:290)
    at org.apache.pig.PigServer.launchPlan(PigServer.java:1475)
    at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1460)
    at org.apache.pig.PigServer.storeEx(PigServer.java:1119)
    at org.apache.pig.PigServer.store(PigServer.java:1082)
    at org.apache.pig.PigServer.openIterator(PigServer.java:995)
    at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:782)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:383)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
    at org.apache.pig.Main.run(Main.java:630)
    at org.apache.pig.Main.main(Main.java:175)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Caused by: java.lang.ClassNotFoundException: org.python.google.common.collect.Lists
    at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    ... 24 more
================================================================================

CodePudding user response：

I think this is a bug in some versions of Pig.

It seems that it (MRJobStats.java) is using org.python.google.common.collect.* classes instead of org.google.common.collect.*, and the former are not on the classpath.

The bug was fixed in this commit:

• https://github.com/apache/pig/commit/6dd3ca4deb84edd9edd7765aa1d12f89a31b1283

in July 2017. Unfortunately, that is after the pig 0.17.0 release that you are using; see https://github.com/apache/pig/blob/trunk/CHANGES.txt.

So you will most likely need to checkout and build pig for yourself. There is a link to the build instructions in the README.txt file on Github: https://github.com/apache/pig

CodePudding user response：

Using the following dependency solves the issue: https://mvnrepository.com/artifact/org.apache.pig/pig/0.17.0-cdh6.3.4