Problem description
1, hadoop cluster deployment on tencent cloud, a master, two slave
2, hadoop can run official wordcount, can the results
3, hadoop version is 3.1.2, hive's version is 3.1.1
4, hive in run select count (1) the from t_test card dead, when in fact t_test only two data in the table
5, hive stuck phenomenon is as follows:

 hive> The set graphs. Job. Reduces=2; 
Hive> The set hive.com pute. Query. Using. Stats=false; 
Hive> Select count (1) the from t_test; 
The Query ID=root_20191130113749_ff62498c - ee7b - 41 a5 - a4e4 - bc3653b074d1 
The Total jobs=1 
Launching Job 1 out of 1 
Number of reduce tasks determined at compile time: 1 
In order to change the business, the load for a reducer (In bytes) : 
Set hive. The exec. Reducers) bytes) per. Reducer=& lt; Number> 
In order to limit the maximum number of reducers: 
The set hive. The exec. Reducers. Max=& lt; Number> 
In order to set a constant number of reducers: 
The set graphs. Job. Reduces=& lt; Number> 

6, in the/TMP/host/hive. Mr Jobs created, found in the log is as follows:
: the 2019-11-30 T11 37:55, 318 WARN [614 f0786 - de9d - 4388-8 fd4-28 ecbf967e14 main] graphs. JobResourceUploader: Hadoop command - line option parsing the not performed. Implement the Tool interface and the execute your application with ToolRunner to remedy this.
The 2019-11-30 T11:37:55, 399 INFO [614 f0786 - de9d - 4388-8 fd4-28 ecbf967e14 main] graphs. JobResourceUploader: Disabling Erasure Coding for path:/TMP/hadoop - yarn/staging/root/. The staging/ job_1575042073347_0015



7, but in http://134.175.61.114:8088/cluster/, but did not find the job

CodePudding user response:

Add:
Hang up to run all night last night, the morning found on http://134.175.61.114:8088/cluster/found to job, information is as follows:

User: root
Name: select count (1) the from t_test (Stage 1)
The Application Type: graphs
Application Tags:
Application Priority: 0 (who Integer value are who Priority)
YarnApplicationState: FAILED
Queue: default
FinalStatus Reported by AM: FAILED
Started: Sat Nov 30 02:13:22 + 0800 2019
Elapsed: 24 mins, 30 SEC
Tracking URL: History
The Log Aggregation Status: DISABLED
Application Timeout (Remaining Time) : Unlimited
Diagnostics: Application application_1575042073347_0009 failed 2 times due to ApplicationMaster for attempt appattempt_1575042073347_0009_000002 timed out. The Failing of the Application. The
Fsound_unmanaged Application: false
Application Node Label expression: & lt; The Not set>
AM a container Node Label expression: & lt; DEFAULT_PARTITION & gt;

CodePudding user response:

If you are interested, I can provide the system account and view more detailed log

CodePudding user response:

A few hours in the past, finally saw the error message:

Hive Session ID=d802bdae - f12c - 4074 - be9e - 9 c425bffed34
Hive> The set graphs. Job. Reduces=2;
Hive> The set hive.com pute. Query. Using. Stats=false;
Hive> Select count (1) the from t_test;
The Query ID=root_20191130113749_ff62498c - ee7b - 41 a5 - a4e4 - bc3653b074d1
The Total jobs=1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the business, the load for a reducer (In bytes) :
Set hive. The exec. Reducers) bytes) per. Reducer=& lt; Number>
In order to limit the maximum number of reducers:
The set hive. The exec. Reducers. Max=& lt; Number>
In order to set a constant number of reducers:
The set graphs. Job. Reduces=& lt; Number>
Starting the Job=job_1575042073347_0015, Tracking URL=http://master:8088/proxy/application_1575042073347_0015/
Kill Command=/opt/hadoop/bin/mapred job - Kill job_1575042073347_0015
Hadoop job information for Stage 1: number of mappers: 0; The number of reducers: 0
The 2019-11-30 12:41:14 map Stage 331-1=0%, reduce 0%=
Ended Job=job_1575042073347_0015 with errors
Error during the job, obtaining was debugging information...
FAILED: Execution Error, return code 2 from org.. Apache hadoop. Hive. Ql. Exec. Mr. MapRedTask
Graphs Jobs Launched:
Stage - Stage 1: HDFS Read: 0 HDFS Write: 0 FAIL
Total graphs of CPU Time Spent: 0 msec
Hive>
A Socket error Event: 32 error: 10053.
Connection closing... The Socket close.

CodePudding user response:

You don't add the front two, executed directly select count (1) the from t_test; Would it be ok?

CodePudding user response:

reference 4 floor RivenDong response:

you don't add the front two, executed directly select count (1) the from t_test; Would it be ok?

Soon returned, but is zero, because read statistics, not MR

CodePudding user response:

reference 5 floor ytoms reply:

Quote: refer to 4th floor RivenDong response:

You don't add the front two, executed directly select count (1) the from t_test; Would it be ok?

Soon returned, but is zero, because read statistics, don't walk MR

Read what is statistics? HQL default will be converted to Mr To perform, you won't have to set itself

CodePudding user response:

nullnullnullnullnullnullnull