Home > other >  Terasort key scheduling problems for Chinese
Terasort key scheduling problems for Chinese

Time:10-04

Each line has a large file is in Chinese or English word hybrid (or key is Chinese), relative to the global order, use the hadoop - example terasort occur when the following error, whether Chinese key sequence to change the implementation of the Trie tree...
14/03/06 16:13:46 INFO terasort. Terasort: starting
14/03/06 16:13:47 INFO mapred. FileInputFormat: Total input paths to the process: 1
14/03/06 16:13:47 INFO zlib. ZlibFactory: Successfully the loaded & amp; An initialized native - zlib library
14/03/06 16:13:47 INFO compress. CodecPool: Got brand - new compressor [. Deflate]
Making 2 from 13 records
The Step size is 6.5
14/03/06 16:13:47 WARN mapred. JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
14/03/06 16:13:47 INFO mapred. JobClient: Running job: job_201403061609_0004
14/03/06 16:13:48 INFO mapred. JobClient: map reduce 0% 0%
14/03/06 16:13:55 INFO mapred. JobClient: map reduce 50% 0%
14/03/06 16:13:57 INFO mapred. JobClient: Task Id: attempt_201403061609_0004_m_000000_0, Status: FAILED
Java. Lang. 23
ArrayIndexOutOfBoundsException: -At org, apache hadoop. Examples. Terasort. Terasort $$InnerTrieNode TotalOrderPartitioner. FindPartition (terasort. Java: 91)
At org, apache hadoop. Examples. Terasort. Terasort $TotalOrderPartitioner. GetPartition (terasort. Java: 221)
At org, apache hadoop. Examples. Terasort. Terasort $TotalOrderPartitioner. GetPartition (terasort. Java: 57)
At org, apache hadoop. Mapred. MapTask $OldOutputCollector. Collect (MapTask. Java: 526)
At org, apache hadoop. Mapred. Lib. IdentityMapper. The map (IdentityMapper. Java: 38)
At org, apache hadoop. Mapred. MapRunner. Run (MapRunner. Java: 50)
At org, apache hadoop. Mapred. MapTask. RunOldMapper (MapTask. Java: 417)
At org, apache hadoop. Mapred. MapTask. Run (332) MapTask. Java:
At org, apache hadoop. Mapred. Child $4. The run (268) Child. Java:
The at Java. Security. The AccessController. DoPrivileged (Native Method) [59/1965]
At javax.mail. Security. Auth. Subject. The doAs (396) Subject. Java:
At org, apache hadoop. Security. UserGroupInformation. DoAs (UserGroupInformation. Java: 1408)
At org, apache hadoop. Mapred. Child. The main (Child
14/03/06 16:14:04 INFO mapred. JobClient: Task Id: attempt_201403061609_0004_m_000000_1, Status: FAILED
Java. Lang. 23
ArrayIndexOutOfBoundsException: -At org, apache hadoop. Examples. Terasort. Terasort $$InnerTrieNode TotalOrderPartitioner. FindPartition (terasort. Java: 91)
At org, apache hadoop. Examples. Terasort. Terasort $TotalOrderPartitioner. GetPartition (terasort. Java: 221)
At org, apache hadoop. Examples. Terasort. Terasort $TotalOrderPartitioner. GetPartition (terasort. Java: 57)
At org, apache hadoop. Mapred. MapTask $OldOutputCollector. Collect (MapTask. Java: 526)
At org, apache hadoop. Mapred. Lib. IdentityMapper. The map (IdentityMapper. Java: 38)
At org, apache hadoop. Mapred. MapRunner. Run (MapRunner. Java: 50)
At org, apache hadoop. Mapred. MapTask. RunOldMapper (MapTask. Java: 417)
At org, apache hadoop. Mapred. MapTask. Run (332) MapTask. Java:
At org, apache hadoop. Mapred. Child $4. The run (268) Child. Java:
The at Java. Security. The AccessController. DoPrivileged (Native Method)
At javax.mail. Security. Auth. Subject. The doAs (396) Subject. Java:
At org, apache hadoop. Security. UserGroupInformation. DoAs (UserGroupInformation. Java: 1408)
At org, apache hadoop. Mapred. Child. The main (Child
14/03/06 16:14:09 INFO mapred. JobClient: Task Id: attempt_201403061609_0004_m_000000_2, Status: FAILED
Java. Lang. ArrayIndexOutOfBoundsException: - 23//
At org, apache hadoop. Examples. Terasort. Terasort $$InnerTrieNode TotalOrderPartitioner. FindPartition (terasort. Java: 91)
At org, apache hadoop. Examples. Terasort. Terasort $TotalOrderPartitioner. GetPartition (terasort. Java: 221)
At org, apache hadoop. Examples. Terasort. Terasort $TotalOrderPartitioner. GetPartition (terasort. Java: 57)
At org, apache hadoop. Mapred. MapTask $OldOutputCollector. Collect (MapTask. Java: 526)
At org, apache hadoop. Mapred. Lib. IdentityMapper. The map (IdentityMapper. Java: 38)
At org, apache hadoop. Mapred. MapRunner. Run (MapRunner. Java: 50)
At org, apache hadoop. Mapred. MapTask. RunOldMapper (MapTask. Java: 417)
At org, apache hadoop. Mapred. MapTask. Run (332) MapTask. Java:
The at Java. Security. The AccessController. DoPrivileged (Native Method) [29/1965]
At javax.mail. Security. Auth. Subject. The doAs (396) Subject. Java:
At org, apache hadoop. Security. UserGroupInformation. DoAs (UserGroupInformation. Java: 1408)
At org, apache hadoop. Mapred. Child. The main (Child
14/03/06 16:14:15 INFO mapred. JobClient: Job complete: job_201403061609_0004
14/03/06 16:14:15 INFO mapred. JobClient: Counters: 29
14/03/06 16:14:15 INFO mapred. JobClient: File System Counters
14/03/06 16:14:15 INFO mapred. JobClient: FILE: Number of bytes read=156
14/03/06 16:14:15 INFO mapred. JobClient: FILE: Number of bytes written=171585
nullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnull
  • Related