www.elastic.co/cn/

es安装

www.elastic.co/guide/en/elasticsea...

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.8.0-linux-x86_64.tar.gz tar -xzf elasticsearch-8.8.0-linux-x86_64.tar.gz cd elasticsearch-8.8.0/ 

启动

./bin/elasticsearch

fatal exception while booting Elasticsearchjava.lang.RuntimeException: can not run elasticsearch as root

mkdir data adduser es passwd es chown -R es:es ./* chmod -R 775 ./* su es

received plaintext http traffic on an https channel, closing connection
ES8默认开启了ssl认证，导致无法访问9200端口
elasticsearch.yml配置:xpack.security.enabled:把true改成false

 curl -X GET "localhost:9200/?pretty"

kibana安装

www.elastic.co/guide/en/kibana/cur...

curl -O https://artifacts.elastic.co/downloads/kibana/kibana-8.8.0-linux-x86_64.tar.gz tar -xzf kibana-8.8.0-linux-x86_64.tar.gz cd kibana-8.8.0/ 

[root@MiWiFi-RA81-srv kibana-8.8.0]# ./bin/kibana
Kibana should not be run as root. Use –allow-root to continue.
[root@MiWiFi-RA81-srv kibana-8.8.0]# ./bin/kibana –allow-root

vim ./config/kibana.yml

#kibana.yml server.port: 5601 server.host: "0.0.0.0" elasticsearch.hosts: ["http://192.168.31.220:9200"]

 ./bin/kibana --allow-root

Logstas安装

www.elastic.co/guide/en/logstash/c...

下载

www.elastic.co/cn/downloads/logsta...

wget https://artifacts.elastic.co/downloads/logstash/logstash-8.8.0-linux-x86_64.tar.gz tar -xzf logstash-8.8.0-linux-x86_64.tar.gz cd logstash-8.8.0

测试启动

bin/logstash -e 'input { stdin { } } output { stdout {} }'

用filebeat发送日志到logstash

filebeat安装

www.elastic.co/guide/en/beats/file...

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.8.0-linux-x86_64.tar.gz tar xzvf filebeat-8.8.0-linux-x86_64.tar.gz cd filebeat-8.8.0-linux-x86_64

配置filebeat

vim filebeat.yml

输入

[root@MiWiFi-RA81-srv log]# pwd /var/log [root@MiWiFi-RA81-srv log]# echo "aaa" > test.log

不输出到es(vim filebeat.yml)

www.elastic.co/guide/en/beats/file...

输出到logstash（不用这个）

www.elastic.co/guide/en/beats/file...

回到logstash文档输出

www.elastic.co/guide/en/logstash/c...

vim logstash-sample.conf

这里输出到终端（下面会改成输出到es） stdout { codec => rubydebug }

input { beats { port => 5044 } } output { stdout { codec => rubydebug } } 

启动logstash（前面没关闭的要关闭，不然报错）

bin/logstash -f ./config/logstash-sample.conf --config.reload.automatic

启动filebeat

www.elastic.co/guide/en/logstash/c...

./filebeat -e -c filebeat.yml -d "publish"

测试

回到logstash窗口，发现输出终端成功

输出到es

[root@MiWiFi-RA81-srv config]# pwd /www/wwwroot/elk/logstash-8.8.0/config [root@MiWiFi-RA81-srv config]# vim logstash-sample.conf 

启动

bin/logstash -f ./config/logstash-sample.conf --config.reload.automatic

测试输出

[root@MiWiFi-RA81-srv log]# pwd /var/log [root@MiWiFi-RA81-srv log]# echo "赛文柒 /admin/login/login" >> test.log 

进入Kibana界面

查询数据

# Click the Variables button, above, to create your own variables. GET logs-generic-default/_search { "query": { "match_all": {} // match_all } }

创建可视化界面（重点）

继续输出日志

根据目录配置es索引

vim filebeat.yml

filebeat.inputs: # Each - is an input. Most options can be set at the input level, so # you can use different inputs for various configurations. # Below are the input specific configurations. # filestream is an input for collecting log messages from files. - type: log # Unique ID among all inputs, an ID is required. #id: my-filestream-id # Change to true to enable this input configuration. enabled: true # Paths that should be crawled and fetched. Glob based paths. paths: - /var/server01/*.log fields: log_type: "server01" - type: log enabled: true paths: - /var/server02/*.log fields: log_type: "server02"

vim ./config/logstash-sample.conf

 input { beats { port => 5044 } } #output { # stdout { codec => rubydebug } #} output { #elasticsearch { # hosts => ["http://192.168.31.220:9200"] # index => "logstash-test-%{index_name}" #user => "elastic" #password => "changeme" # } if [fields][log_type] == "server01" { elasticsearch { hosts => ["http://192.168.31.220:9200"] index => "server01-%{+YYYY.MM.dd}" } } if [fields][log_type] == "server02" { elasticsearch { hosts => ["http://192.168.31.220:9200"] index => "server02-%{+YYYY.MM.dd}" } } }

参考Filebeat+Logstash采集多个日志文件写入不同的ES索引

filebeat 配置多行解析

filebeat.inputs: # Each - is an input. Most options can be set at the input level, so # you can use different inputs for various configurations. # Below are the input specific configurations. # filestream is an input for collecting log messages from files. - type: log # Unique ID among all inputs, an ID is required. #id: my-filestream-id # Change to true to enable this input configuration. enabled: true # Paths that should be crawled and fetched. Glob based paths. paths: - /var/server01/*.log multiline.pattern: '^\d{4}-\d{2}-\d{2}' multiline.negate: true multiline.match: after fields: log_type: "server01" - type: log enabled: true paths: - /var/server02/*.log multiline.pattern: '^\d{4}-\d{2}-\d{2}' multiline.negate: true multiline.match: after fields: log_type: "server02" 

脚本运行

vim elk.sh

 #!/bin/bash export ES_HOME=/www/wwwroot/elk/elasticsearch-8.8.0/ export KIBANA_HOME=/www/wwwroot/elk/kibana-8.8.0/ export LOGSTASH_HOME=/www/wwwroot/elk/logstash-8.8.0/ export FILEBEAT_HOME=/www/wwwroot/elk/filebeat-8.8.0-linux-x86_64/ su es <<EOF cd $ES_HOME pwd nohup ./bin/elasticsearch & echo "es start" EOF cd $KIBANA_HOME pwd nohup ./bin/kibana --allow-root & echo "kibana start" cd $LOGSTASH_HOME pwd nohup bin/logstash -f ./config/logstash-sample.conf --config.reload.automatic & echo "logstash start" cd $FILEBEAT_HOME pwd nohup ./filebeat -e -c filebeat.yml -d "publish" & echo "filebeat start" 

启动

./elk.sh

vim elkstop.sh

#!/bin/bash pid=$(netstat -nlp | grep 9200 | awk '{print $7}' | awk -F"/" '{ print $1 }'); if [ -n "$pid" ]; then kill -9 $pid; fi echo "es is stopped" pid=$(netstat -nlp | grep 5601 | awk '{print $7}' | awk -F"/" '{ print $1 }'); if [ -n "$pid" ]; then kill -9 $pid; fi echo "kibana is stopped"

关闭

./elkstop.sh

补充

nohup后台运行可以输入fg退出到终端

注意防火墙端口

效果图

本作品采用《CC 协议》，转载必须注明作者和本文链接