# # in order to report errors in the log write

WARN [conf - file - the poller - 0] (org. Apache. The flume. Conf. FlumeConfiguration $AgentConfiguration. ValidateConfigFilterSet, 623) - the Agent configuration for 'a2' has no configfilters.



ERROR [conf - file - the poller - 0] (org. Apache. The flume. Node. AbstractConfigurationProvider. LoadSinks, 469) - Sink k2 has had been removed due to an ERROR during configuration


ERROR [conf - file - the poller - 0] (org. Apache. The flume. Node. AbstractConfigurationProvider. LoadSinks, 469) - Sink k2 has had been removed due to an ERROR during configuration
Java. Lang. InstantiationException: Incompatible sink and channel Settings defined. The sink 's batch size is greater than the channels transaction capacity. The sink: k2, batch size=1000, channel c2, transaction capacity=100
The at org. Apache. The flume. Node. AbstractConfigurationProvider. CheckSinkChannelCompatibility (AbstractConfigurationProvider. Java: 403)
The at org. Apache. The flume. Node. AbstractConfigurationProvider. LoadSinks (AbstractConfigurationProvider. Java: 462)
The at org. Apache. The flume. Node. AbstractConfigurationProvider. GetConfiguration (AbstractConfigurationProvider. Java: 106)
The at org. Apache. The flume. Node. PollingPropertiesFileConfigurationProvider $FileWatcherRunnable. Run (PollingPropertiesFileConfigurationProvider. Java: 145)
The at Java. Util. Concurrent. Executors $RunnableAdapter. Call (511) Executors. Java:
The at Java. Util. Concurrent. FutureTask. RunAndReset (FutureTask. Java: 308)
The at Java. Util. Concurrent. ScheduledThreadPoolExecutor $ScheduledFutureTask. Access the $301 (180) ScheduledThreadPoolExecutor. Java:
The at Java. Util. Concurrent. ScheduledThreadPoolExecutor $ScheduledFutureTask. Run (ScheduledThreadPoolExecutor. Java: 294)
The at Java. Util. Concurrent. ThreadPoolExecutor. RunWorker (ThreadPoolExecutor. Java: 1142)
The at Java. Util. Concurrent. ThreadPoolExecutor $Worker. The run (ThreadPoolExecutor. Java: 617)
The at Java. Lang. Thread. The run (Thread. Java: 745)

# o great god answers to this configuration file is
A2. Sources r2=
A2. Sinks=k2
A2. Channels=c2

# the Describe/configure the source
A2. Sources. R2. Type=exec
A2.sources.r2.com mand=tail -f/usr/packge/hive/hive - 1.1.0 - cdh5.7.0/log

# the Describe the sink
A2. Sinks. K2. Type=HDFS
A2. Sinks. K2. HDFS. Path=HDFS://master: 9000/flume/% % % m % d/H Y

A2. Sinks. K2. HDFS. FilePrefix=logs -

A2. Sinks. K2. HDFS. Round=true

A2. Sinks. K2. HDFS. RoundValue=https://bbs.csdn.net/topics/1

A2. Sinks. K2. HDFS. RoundUnit=hour

A2. Sinks. K2. HDFS. UseLocalTimeStamp=true

A2. Sinks. K2. HDFS. BatchSize=1000

A2. Sinks. K2. HDFS. FileType=DataStream

A2. Sinks. K2. HDFS. RollInterval=30

A2. Sinks. K2. HDFS. RollSize=134217700

A2. Sinks. K2. HDFS. RollCount=0


A2. Channels. C2. Type=memory
A2. Channels. C2. Capacity=1000
A2. Channels. C2. TransactionCapacity=100


A2. Sources. R2. Channels=c2
A2. Sinks. K2. Channel=c2

CodePudding user response:

A2. Sinks. K2. HDFS. BatchSize=1000
The value set here is too big, HDFS. BatchSize is not greater than the following transactionCapacity
Change
A2. Sinks. K2. HDFS. BatchSize=200
A2. Channels. C2. TransactionCapacity=300
It's good