1. The spark - 2.1.1 - bin - hadoop2.7, CentOS 7, the high availability cluster based on Standalone installation, a total of five nodes,
2. My question is, in each node SPARK_HOME/logs under several log file is very large, and not by size or date scroll: spark - admin - org. Apache. Spark. Deploy. Worker. The worker - 1 - HOST_NAME. Out, spark - admin - org. Apache. Spark. Deploy. Master. The master - 1 - HOST_NAME. Out, spark - admin - org. Apache. Spark. Deploy. History. HistoryServer - 1 - HOST_NAME. Out, the level of a lot of DEBUG level is, extract a few lines as follows:
The 20-01-20. 10:27:56. [27] WorkerUI - 704 the DEBUG SelectChannelEndPoint - Local interests updating 0 - & gt; 1 for 7 dedb6f1 SelectChannelEndPoint @ {/10.221.216.68:43512 & lt; - & gt; 8081, the Open, in OSHUT, R, -, 0/30000, HttpConnection} {IO=1, kio=0, kro=1}
The 20-01-20. 10:27:56. [27] WorkerUI - 704 the DEBUG SelectorManager - Queued change org. Spark_project. Jetty. IO. SelectChannelEndPoint $1 @ 64 e3a4da
20-01-20. 10:27:56. [WorkerUI 704-25 - the selector - ServerConnectorManager @ 1 a0e7661/1] DEBUG SelectorManager - the selector loop woken up from the select, 0/1 selected
20-01-20. 10:27:56. [WorkerUI 704-25 - the selector - ServerConnectorManager @ 1 a0e7661/1] DEBUG SelectorManager - Running change org. Spark_project. Jetty. IO. SelectChannelEndPoint $1 @ 64 e3a4da
20-01-20. 10:27:56. [WorkerUI 704-25 - the selector - ServerConnectorManager @ 1 a0e7661/1] DEBUG SelectChannelEndPoint - Key interests updated 0 - & gt; 1 on SelectChannelEndPoint @ 7 dedb6f1 {/10.221.216.68:43512 & lt; - & gt; 8081, the Open, in OSHUT, R, -, 0/30000, HttpConnection} {IO=1, kio=1, kro=1}
20-01-20. 10:27:56. [WorkerUI 704-25 - the selector - ServerConnectorManager @ 1 a0e7661/1] DEBUG SelectorManager - the selector loop waiting on the select
20-01-20. 10:27:56. [WorkerUI 707-25 - the selector - ServerConnectorManager @ 1 a0e7661/1] DEBUG SelectorManager - the selector loop woken up from the select, 1/1 selected
20-01-20. 10:27:56. [WorkerUI 708-25 - the selector - ServerConnectorManager @ 1 a0e7661/1] DEBUG SelectChannelEndPoint - Key interests updated 1 - & gt; 0 on SelectChannelEndPoint @ 7 dedb6f1 {/10.221.216.68:43512 & lt; - & gt; 8081, the Open, in OSHUT, R, -, 3/30000, HttpConnection} {IO=1, kio=0, kro=1}
20-01-20. 10:27:56. [WorkerUI 708-25 - the selector - ServerConnectorManager @ 1 a0e7661/1] DEBUG SelectChannelEndPoint - Local interests updating 1 - & gt; 0 for 7 dedb6f1 SelectChannelEndPoint @ {/10.221.216.68:43512 & lt; - & gt; 8081, the Open, in OSHUT, R, -, 4/30000, HttpConnection} {IO=0, kio=0, kro=1}
20-01-20. 10:27:56. [WorkerUI 708-25 - the selector - ServerConnectorManager @ 1 a0e7661/1] DEBUG SelectorManager - Queued change org. Spark_project. Jetty. IO. SelectChannelEndPoint $1 @ 64 e3a4da
20-01-20. 10:27:56. [WorkerUI 708-25 - the selector - ServerConnectorManager @ 1 a0e7661/1] DEBUG AbstractConnection - FILL_INTERESTED - & gt; FILLING HttpConnection @ 74 a9ce80 [FILLING, 7 dedb6f1 SelectChannelEndPoint @ {/10.221.216.68:43512 & lt; - & gt; 8081, the Open, in, OSHUT, -, -, 4/30000, HttpConnection} {IO=0, kio=0, kro=1}] [p=HttpParser of 0} {s=CLOSED, 0, g=HttpGenerator=START} {s, c=HttpChannelOverHttp @ 743 ee72f {r=1, c=false, a=IDLE, uri=}]
20-01-20. 10:27:56. [WorkerUI 708-25 - the selector - ServerConnectorManager @ 1 a0e7661/1] DEBUG SelectorManager - Running change org. Spark_project. Jetty. IO. SelectChannelEndPoint $1 @ 64 e3a4da
20-01-20. 10:27:56. [WorkerUI 708-25 - the selector - ServerConnectorManager @ 1 a0e7661/1] DEBUG SelectorManager - the selector loop waiting on the select
20-01-20. 10:27:56. [30] WorkerUI - 708 the DEBUG HttpConnection - HttpConnection @ 74 a9ce80 [FILLING, SelectChannelEndPoint @ 7 dedb6f1 {/10.221.216.68:43512 & lt; - & gt; 8081, the Open, in, OSHUT, -, -, 4/30000, HttpConnection} {IO=0, kio=0, kro=1}] [p=HttpParser of 0} {s=CLOSED, 0, g=HttpGenerator=START} {s, c=HttpChannelOverHttp @ 743 ee72f {r=1, c=false, a=IDLE, uri=}] onFillable HttpChannelState @ 485 a5da2 {s=IDLE I=true a=null}
The 20-01-20. 10:27:56. [30] WorkerUI - 708 the DEBUG ChannelEndPoint - filled - 1 SelectChannelEndPoint @ 7 dedb6f1 {/10.221.216.68:43512 & lt; - & gt; 8081, the Open, in OSHUT, -, -, 4/30000, HttpConnection} {IO=0, kio=0, kro=1}
The 20-01-20. 10:27:56. [30] WorkerUI - 708 the DEBUG ChannelEndPoint - ishut SelectChannelEndPoint @ 7 dedb6f1 {/10.221.216.68:43512 & lt; - & gt; 8081, the Open, in OSHUT, -, -, 4/30000, HttpConnection} {IO=0, kio=0, kro=1}
The 20-01-20. 10:27:56. [30] WorkerUI - 708 the DEBUG AbstractEndPoint - onClose SelectChannelEndPoint @ 7 dedb6f1 {/10.221.216.68:43512 & lt; - & gt; 8081, CLOSED, ISHUT OSHUT, -, -, 4/30000, HttpConnection} {IO=0, kio=0, kro=1}
The 20-01-20. 10:27:56. [30] WorkerUI - 708 the DEBUG ChannelEndPoint - close SelectChannelEndPoint @ 7 dedb6f1 {/10.221.216.68:43512 & lt; - & gt; 8081, CLOSED, ISHUT OSHUT, -, -, 4/30000, HttpConnection} {IO=0, kio=0, kro=1}
The 20-01-20. 10:27:56. [30] WorkerUI - 708 the DEBUG SelectorManager - Destroyed SelectChannelEndPoint @ 7 dedb6f1 {/10.221.216.68:43512 & lt; - & gt; 8081, CLOSED, ISHUT OSHUT, -, -, 4/30000, HttpConnection} {IO=0, kio=1, kro=1}
nullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnull