Use of regular expressions to know deep, logic structure need to be clear, practice need learning and why, just can achieve mastery through a comprehensive study!
A. Use Kibana
1. Kibana is: data visualization platform tools
2. Features:
Flexible analysis and visualization platform
Real-time traffic and data chart
Display intuitive interface for different user
In real-time and embedded dashboard
3. Kibana installation: yum - y install kibana
4. Default installed at/opt/kibana kibana below, configuration files in/opt/kibana/config/kibana yml
5. Kibana. Yml configuration:
. 2 for server port: 5601
5 for server host: "0.0.0.0" 15 elasticsearch. Url: "http://192.168.1.51:9200" is 23 kibana. Index: "kibana" 26 kibana. DefaultAppId: "discover" 53 elasticsearch. PingTimeout: 1500
57 elasticsearch. RequestTimeout: 30000
64 elasticsearch. StartupTimeout: 5000
6. Start the service and set up the boot from the rev.
Systemctl start kibana
Systemctl enable kibana
7. Web access kibana: http://192.168.1.56:5601/
After 8. Start the service, the web page will automatically generate kibana database cluster
2. Logstash config extensions
1. The Logstash: a data collection, processing and transmission tool
2. Features:
All types of centralized data processing
Different mode and the format of the data normalization
Custom log format of rapidly expanding
For custom data source adding plugin
3. The Logstash installation
Depends on (1). Install Java environment: yum -y install Java -- 1.8.0 comes with its
(2). Install package: yum - y install logstash
(3) installed at/opt/logstash directory
4. The Logstash work structure:
{source}==& gt;
Input {}==& gt;
The filter==& gt;
The output {}==& gt;
{ES}
Input to collect data is sent to the filter, filter processing data to the output, the output data forwarding out
5. The inside of the Logstash type
Boolean value type: ssl_enable=& gt; True
Byte type: bytes=& gt; "1 mib" string type: name=& gt; "Xkops" numeric types: port=& gt; 22
Array: match=& gt; [" datetime ", "UNIX"]
Hash: options=& gt; {k=& gt; "V", k2=& gt; "V2"}
Encoding decoding: codec=& gt; "Json" path: file_path=& gt; "/TMP/filename" note: #
6. Logstash condition judgment
Is equal to:==is not equal to:!=less than: & lt; More than: & gt; Less than or equal to: & lt;=is greater than or equal to: & gt;=matches a regular:=~ don't match regular:! ~
7. Logstash condition judgment
Contains: does not contain: in the not with: in the and, or, or not with: nand or not: xor
Compound expressions: take anticoincidence: ()! ()
8. Logstash plug-in:/opt/Logstash/bin/Logstash - the plugin list # view of plug-in
9. The plugin
Codec class plug-in (common) : plain, json, json_lines, rubydebug, multiline
The input file plugin
The path - & gt; The specified log file path
Type - & gt; To the specified log file tag
Start_position - & gt; Default mode, first read configuration files from where
Sincedb_path - & gt; The position of the pointer file path, the record read the file
11. The filter grok plug-in:
Resolution of log data of unstructured plug-in
Grok used regular expressions to unstructured data structured
In the group match, a regular expression to write according to the specific data structure
Writing is difficult, but a wide applicability
3. Filebeat installation configuration
1. Filebeat installation: yum - y install filebeat
2. Modify the configuration file:/etc/filebeat/filebeat yml
3. The open service, and set up the boot operation:
Systemctl start filebeat
Systemctl enable filebeat
