Problem description:
Input two files:
The first file contents: Hello World Bye World
The second file contents: Hello Hadoop Goodbye Hadoop are ate
Run WordCount program output content: is the above two lines!
Hello Hadoop Goodbye Hadoop are ate
Hello World Bye World
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
WordCount program should be no problem, should be running normally before, today suddenly runs abnormally, feel like Mapper, Reducer don't work, really don't know what happened!

Wanted to see if the hadoop process running normally, but after entering the installation directory for the hadoop bin folder, it should say the JPS command to find!


Console to print some information:
13/10/02 21:35:43 WARN util. NativeCodeLoader: Unable to load the native - hadoop library for your platform... Using the builtin - Java classes where applicable
13/10/02 21:35:43 WARN mapred. JobClient: No job jar file set. The User classes may not be found. See JobConf (Class) or JobConf# setJar (String).
13/10/02 21:35:43 INFO input. FileInputFormat: Total input paths to the process: 2
13/10/02 21:35:43 WARN snappy. LoadSnappy: after native library not the loaded
13/10/02 21:35:44 INFO mapred. JobClient: Running job: job_local255489033_0001
13/10/02 21:35:44 INFO mapred. LocalJobRunner: Waiting for the map tasks
13/10/02 21:35:44 INFO mapred. LocalJobRunner: Starting task: attempt_local255489033_0001_m_000000_0
13/10/02 21:35:44 INFO util. ProcessTree: setsid exited with exit code 0
13/10/02 21:35:44 INFO mapred. Task: Using ResourceCalculatorPlugin: org.. Apache hadoop. Util. 58 dad8b5 LinuxResourceCalculatorPlugin @
13/10/02 21:35:44 INFO mapred. MapTask: Processing the split: HDFS://localhost: 9000/user/test/input/2. TXT: 0 + 36
13/10/02 21:35:44 INFO mapred. MapTask: IO. Sort. MB=100
13/10/02 21:35:45 INFO mapred. MapTask: data buffer=79691776/99614720
13/10/02 21:35:45 INFO mapred. MapTask: record buffer=262144/327680

CodePudding user response:

The source code posted to see

CodePudding user response:

reference 1st floor s060403072 response:

source posted see?

Package testpkg;

Import the Java. IO. IOException;
Import the Java. Util. StringTokenizer;

Import org, apache hadoop. Conf. Configuration;
Import org, apache hadoop. Fs. The Path;
Import org, apache hadoop. IO. IntWritable;
Import org, apache hadoop. IO. Text;
Import org, apache hadoop. Graphs. The Job;
Import org, apache hadoop. Graphs. Mapper;
Import org, apache hadoop. Graphs. Reducer;
Import org, apache hadoop. Graphs. Lib. Input. FileInputFormat;
Import org, apache hadoop. Graphs. Lib. Output. FileOutputFormat;
Import org, apache hadoop. Util. GenericOptionsParser;

Public class WordCount {

Public static class TokenizerMapper
Extends Mapper {

Private final static IntWritable one=new IntWritable (1);
Private Text word=new Text ();

Public void map (Object key, the Text value, Context Context
) throws IOException, InterruptedException {
StringTokenizer itr=new StringTokenizer (value. The toString ());
While (itr. HasMoreTokens ()) {
Word. The set (itr. NextToken ());
The context. Write (word, one);
}
}
}

Public static class IntSumReducer
Extends Reducer {
Private IntWritable result=new IntWritable ();

Public void the reduce (Text key Iterable Values,
The Context Context
) throws IOException, InterruptedException {
Int sum=0;
For (IntWritable val: values) {
The sum +=val. The get ();
}
The result set (sum);
Context. Write (key, result);
}
}

Public static void main (String [] args) throws the Exception {
Configuration conf=new Configuration();
String [] otherArgs=new GenericOptionsParser (conf, args). GetRemainingArgs ();
If (otherArgs length!=2) {
System. Err. Println (" the Usage: wordcount & lt; In> " );
System. The exit (2);
}
Job Job=new Job (conf, "word count");
Job. SetJarByClass (WordCount. Class);
Job. SetMapperClass (TokenizerMapper. Class);
Job. SetCombinerClass (IntSumReducer. Class);
Job. SetReducerClass (IntSumReducer. Class);

Job. SetOutputKeyClass (Text. Class);
Job. SetOutputValueClass (IntWritable. Class);

FileInputFormat. AddInputPath (job, new Path (otherArgs [0]));
FileOutputFormat. SetOutputPath (job, new Path (otherArgs [1]));
System. The exit (job. WaitForCompletion (true)? 0:1);
}
}

CodePudding user response:

Try again to restart the machine

CodePudding user response:

reference s060403072 reply: 3/f

can restart the machine try again

Restart the wow, is that, forget it, I'll try to reconfigure the ~

CodePudding user response:

Well, the feeling is the issue of environment variables,,,

CodePudding user response:

JSP not make? That's Java environment problem,/etc/profile configuration is wrong, and hadop - env. Sh also get set javahome,

CodePudding user response:

The JPS command is in JDK bin directorynullnullnullnull