-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - line -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Has now set up complete HDFS + YARN and Flink independent cluster, the host is as follows:
Flink independent cluster:
F - node - 01
F - node - 02
Hadoop cluster:
H - name - 01
H - name - 02
H - data - 01
H - data - 02
Using Flink independent cluster how to use is no problem, the tested using, now want the Flink submitted to YARN in the cluster, adopt the way of Flink On YARN to run the Flink task, select independent Flink Job way from f - node - submit tasks to h - 01 node name - 01 YARN node, have YARN - site. XML copy to f - node - 01 nodes. At the same time configuration YARN_CONF_DIR environment variables, but in the executive [./Flink run - m YARN - cluster - yjm 2048 m - 2048 m/TMP/ytm Flink - demo. Jar] command, suggests the following error:
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Under Caused by: org. Apache. Flink. Yarn. YarnClusterDescriptor $YarnDeploymentException: The yarn application unexpectedly switched to The state FAILED during deployment.
Diagnostics from YARN: Application application_1603786544958_0001 failed 2 times in the previous 10000 milliseconds due to AM Container for appattempt_1603786544958_0001_000002 exited with exitCode: - 1000
Failing this attempt. Diagnostics: the File File:/root/flink application_1603786544958_0001/lib/flink - table_2. 11-1.11.2. Jar does not exist
Java. IO. FileNotFoundException: File File:/root/flink application_1603786544958_0001/lib/flink - table_2. 11-1.11.2. Jar does not exist
At org, apache hadoop. Fs. RawLocalFileSystem. DeprecatedGetFileStatus (RawLocalFileSystem. Java: 635)
At org, apache hadoop. Fs. RawLocalFileSystem. GetFileLinkStatusInternal (RawLocalFileSystem. Java: 861)
At org, apache hadoop. Fs. RawLocalFileSystem. GetFileStatus (RawLocalFileSystem. Java: 625)
At org, apache hadoop. Fs. FilterFileSystem. GetFileStatus (FilterFileSystem. Java: 442)
At org, apache hadoop. Yarn. Util. FSDownload. Copy (253) FSDownload. Java:
At org, apache hadoop. Yarn. Util. FSDownload. Access the $000 (63) FSDownload. Java:
At org, apache hadoop. Yarn. Util. FSDownload $2. The run (361) FSDownload. Java:
At org, apache hadoop. Yarn. Util. FSDownload $2. The run (359) FSDownload. Java:
The at Java. Security. The AccessController. DoPrivileged (Native Method)
At javax.mail. Security. Auth. Subject. The doAs (422) Subject. Java:
At org, apache hadoop. Security. UserGroupInformation. DoAs (UserGroupInformation. Java: 1836)
At org, apache hadoop. Yarn. Util. FSDownload. Call (358) FSDownload. Java:
At org, apache hadoop. Yarn. Util. FSDownload. Call (62) FSDownload. Java:
The at Java. Util. Concurrent. FutureTask. Run (FutureTask. Java: 266)
The at Java. Util. Concurrent. Executors $RunnableAdapter. Call (511) Executors. Java:
The at Java. Util. Concurrent. FutureTask. Run (FutureTask. Java: 266)
The at Java. Util. Concurrent. ThreadPoolExecutor. RunWorker (ThreadPoolExecutor. Java: 1149)
The at Java. Util. Concurrent. ThreadPoolExecutor $Worker. The run (ThreadPoolExecutor. Java: 624)
The at Java. Lang. Thread. The run (Thread. Java: 748)
For more detailed output, check the application tracking page: http://h-name-01:8088/cluster/app/application_1603786544958_0001, Then click on the links to logs of each attempt.
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Could you tell me the question is what causes?
CodePudding user response:
Can see the cause of the problem YARN container is can't find the Flink JAR as a result, but how to make YARN can be used in the Flink published to the YARN on the automatic upload Flink JAR?CodePudding user response:
Under the great god can help? Or provide some empirical Suggestions and see if I was in the process of yarn on what's missing key steps?
CodePudding user response:
