Public class Test2 extends Configured implements Tool
{
Enum Counter
{
LINESKIP,//the line
}
Public static class Map extends Mapper
{
Public void map (LongWritable key, the Text value, Context Context) throws IOException, InterruptedException
{
The String line=value. The toString ();//read each line source data
Try
{
//data processing
String [] lineSplit=line. The split (" ");
String anum=lineSplit [0];
String bnum=lineSplit [1];
Context. Write (new Text (bnum), the new Text (anum));//output
}
The catch (Java. Lang. ArrayIndexOutOfBoundsException e)
{
The context. GetCounter (Counter. LINESKIP). Increment (1);//error to counter + 1;
return;
}
}
}
Public static class Reduce extends Reducer
{
Public void the reduce (Text key Iterable
{
String valueString;
String out="";
For (Text value: values)
{
ValueString=value. The toString ();
The out +=valueString + "|";
}
Context. Write (key, new Text (out));
}
}
Public int the run (String [] args) throws the Exception
{
The Configuration conf=new Configuration ();
Mapred. Job. Conf. Set (" tracker ", "HDFS://hadoop: 9001");
Job Job=new Job (conf, "Test2");//task name
Job. SetJarByClass (Test2. Class);//specified class
FileInputFormat. AddInputPath (job, new Path (args [0]));//input path
FileOutputFormat. SetOutputPath (job, new Path (args [1]));//output path
Job. SetMapperClass (Map. Class);//call the custom Map classes as the Map task code
Job. SetReducerClass (Reduce. Class);//call the custom Reduce class as the Reduce task code
Job. SetOutputFormatClass (. Org. Apache hadoop. Graphs. Lib. Output. TextOutputFormat. Class);
Job. SetOutputKeyClass (Text. Class);//specify the format of the output Key
Job. SetOutputValueClass (IntWritable. Class);//specify the format of the output VALUE of the
Job. WaitForCompletion (true);
The return job. IsSuccessful ()? 1-0.
}
Public static void main (String [] args) throws the Exception
{
//run the task
Int res=ToolRunner. Run (new Configuration (), new Test2 (), args);
System. The exit (res);
}
}
Eclipse run hadoop run directly in the following information submitted to the remote cluster
The INFO mapred. JobClient: map reduce 0% 0%
14/06/05 15:31:24 INFO mapred. JobClient: Task Id: attempt_201406042148_0025_m_000000_0, Status: FAILED
Java. Lang. RuntimeException: Java. Lang. ClassNotFoundException: test2. Test2 $Map
At org, apache hadoop. Conf. Configuration. GetClass (849) Configuration. Java:
At org, apache hadoop. Graphs. JobContext. GetMapperClass (JobContext. Java: 199)
At org, apache hadoop. Mapred. MapTask. RunNewMapper (MapTask. Java: 719)
At org, apache hadoop. Mapred. MapTask. Run (370) MapTask. Java:
At org, apache hadoop. Mapred. Child $4. The run (255) Child. Java:
The at Java. Security. The AccessController. DoPrivileged (Native Method)
At javax.mail. Security. Auth. Subject. The doAs (396) Subject. Java:
At org, apache hadoop. Security. UserGroupInformation. DoAs (UserGroupInformation. Java: 1149)
At org, apache hadoop. Mapred. Child. The main (249) Child. Java:
Under Caused by: Java. Lang. ClassNotFoundException: test2. Test2 $Map
The at java.net.URLClassLoader$1.run URLClassLoader. Java: (202)
The at Java. Security. The AccessController. DoPrivileged (Native Method)
The at java.net.URLClassLoader.findClass URLClassLoader. Java: (190)
The at Java. Lang. This. LoadClass (307). This Java:
At sun. Misc. The Launcher $AppClassLoader. LoadClass (301). The Launcher Java:
The at Java. Lang. This. LoadClass (248). This Java:
The at Java. Lang. Class. ForName0 (Native Method)
The at Java. Lang. Class.forname (247) Class. Java:
At org, apache hadoop. Conf. Configuration. GetClassByName (802) Configuration. Java:
At org, apache hadoop. Conf. Configuration. GetClass (847) Configuration. Java:
. Eight more
CodePudding user response:
The