A. ELK (deployment)

Elasticsearch - & gt; Database - & gt; Mysql
Effect: the storage and retrieval of data
Write data logstash -> PHP
From the Web log cases of collecting and processing data and stored in E
Kibana - & gt; Nginx
Read data from the E and visual display function: distributed centralized logging data query and management, monitoring, troubleshooting, security information and event management and reporting features. * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Traffic leapt:
Thinking:
- with Zabbix traffic spikes in where?
- determine which system specific see again after a system log analysis reason. The solution:
DDOS -> flow cleaning (to find operators)
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Elasticksearch
Features:
- real-time analysis;
- distributed real-time file storage, and each ES with relational database
1.
Library? - table? - line? - column
Index - type - documentation - domain (field) 2.
MySQL: using SQL to add and delete
Es: using the HTTP protocol to add and delete
Single ES deployment (9300) : 9200, :
Environment for + installation deployment idea:
- host name resolution
- dependent package (Java development, so its, minimum Java 7)
- the installation package
- modify configuration files
To start the service
- check service cluster configuration (based on the single, 5)
- stop service
- modify configuration:
Cluster. Name:??????????? # to identical
Node. Name:?????????? # the machine host name
Network. The host: 0.0.0.0???????? # listening address
Discovery. Zen. Ping. Unicast. Hosts: [" es1 ", "es2", "es3"]
- each node to ping each other (host name resolution)
- have a Java environment
- start the service _cluster/health? Pretty
ES the use of plug-in
- head plug-in
The cluster topology, indexes and the operation of the node level;
Provides query API, and will result in Json and returns form.
- kopf plug-in
Management tool to ES - is
- provide on ES cluster operation API
- bigdesk plug-in
- ES a cluster monitoring tool
- can be through which to view the various states of ES cluster. Such as: CPU, memory, usage, index data, search, the number of HTTP connections. RESTful API
The HTTP request is divided into three parts:
- request line, message headers, request body.
Request lines begin with a method of symbol, separated by a space, followed by the request URI and protocol version, format is as follows:
Method the Request URI HTTP - Version CRLF HTTP Request Method (1.1/8; 1.0/3)
Commonly used: GET, POST, HEAD
ES are commonly used:
- PUT??? Add
- DELETE??? Delete
- POST??? Change
- GET??? Check the curl command:
Is to use the URL rules, work under command line text transport tools, support for multiple requests patterns, custom request the first class power.
- A? Modify the request agent
- X? Set the request method
-i? Show return header
RESTful API simple practical
_cat below API information see
The health to track the status of the cluster, node information
The curl http://es2:9200/_cat/health (? (v)? Help)? # see help
Nodes view node status information
The curl http://es1:9200/_cat/nodes
Indices index information
Curl http://es3:9200/_cat/indices use the curl command, calling RESTful API for elasticsearch to add and delete
Call way: database address/index/type/id value # curl - d??????? # is POST HTTP data options
- increase: PUT
The curl - XPUT 'http://es1:9200/tedu/teacher/1' - d
'{" setting ": {}" index ":" number_of_shards ": 5" number_of_replicas ": 1}
'- DELETE: DELETE
The curl - XDELETE http://es1:9200/tedu/teacher/1
- change: POST
The curl - XPOST http://es1:9200/tedu/teacher/1/_update '{doc: {" name ":" elder brother pillar "}}
'- check: GET
The curl - XGET _bulk batch import data
http://es1:9200/tedu/teacher/1Curl - XPOST http://es1:9200/_bulk - data - binary @ logs. Jsonl added:
Apache log analysis:
127.0.0.1 - [10/Jan/2020:09:40:06 + 0800] "the GET/HTTP/1.1" 12 ", "200" curl/7.29.0 "source IP - E - mail - page user authentication login name - time - access to the resource - request - request way agreement - the server returned a status code - return to the page size indicate which page to submit the request is over - client access tool