最新ELK日志分析系统搭建
环境：centos6.8
准备好ELK三个安装包，到官网下
官网 https://www.elastic.co/
https://s3.运维网.com/wyfs02/M02/8D/8C/wKioL1ihZybjMr4jAAAYV-WXajs416.png-wh_500x0-wm_3-wmp_4-s_1000325584.png
1、安装elasticsearch
这里安装1.8版本的
https://s5.运维网.com/wyfs02/M00/8D/8C/wKioL1ihZ3DD7luTAAAMwgM1LqI347.png-wh_500x0-wm_3-wmp_4-s_1571179791.png
2、安装elasticsearch
下载安装包（tar）https://www.elastic.co/downloads/elasticsearch
直接解压到/usr/local下面
tar -xzf elasticsearch-5.2.0.tar.gz -C /usr/local/
这个版本的elasticsearch跟之前的启动方式不一样了，因为新版的是不允许使用root用户启动了得
我们先新建一个用户elk
useradd elk
然后授权
chown -R elk:elk /usr/local/elasticsearch-5.2.0/
最后切换到elk用户启动
nohup /usr/local/elasticsearch-5.2.0/bin/elasticsearch &
最后检查启动状态，如图所示就对了
# curl 127.0.0.1:9200https://s5.运维网.com/wyfs02/M02/8D/8C/wKioL1ihaD-w6hxOAAAgyi2PKyQ202.png-wh_500x0-wm_3-wmp_4-s_3024959614.png
3、安装logstash
解压
# tar -xzf logstash-5.2.0.tar.gz -C /usr/local/
编辑配置文件
# cat /usr/local/logstash-5.2.0/config/nginx.ymlinput {
beats { #监听在5043端口接收来自filebeat的日志
port => "5043"
}
}
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}"} #过滤规则
}
geoip {
source => "clientip" #过滤规则获取IP
}
}
output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}
启动
nohup /usr/local/logstash-5.2.0/bin/logstash -f /usr/local/logstash-5.2.0/config/nginx.yml & 4、安装filebeat
在客户端安装filebeat，用于推送日志
# tar -xzf filebeat-5.2.0-linux-x86_64.tar.gz -C /usr/local/
新建推送配置
vim /usr/local/filebeat-5.2.0-linux-x86_64/ipaper.ymlfilebeat.prospectors:
- input_type: log
paths:
- /data/wwwlogs/test1.log #指定推送日志文件
- /data/wwwlogs/test2.log
output.logstash:
hosts: ["192.168.0.54:5043"] #指定接收logstash
启动filebeat
# nohup /usr/local/filebeat-5.2.0-linux-x86_64/filebeat -e -c /usr/local/filebeat-5.2.0-linux-x86_64/ipaper.yml -d "publish" &# tail -20 nohup.out
"input_type": "log",
"message": "119.147.33.18 - - \"GET /29204.htm HTTP/1.1\" 200 14344 \"http://epaper.oeeee.com/epaper/M/html/2016-12/06/content_101411.htm\" \"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)\"",
"offset": 44870189,
"source": "/data/wwwlogs/test.log",
"type": "log"
}
2017/02/13 09:39:44.899627 client.go:184: DBGPublish: {
"@timestamp": "2017-02-13T09:39:32.116Z",
"beat": {
"hostname": "ND31",
"name": "ND31",
"version": "5.2.0"
},
"input_type": "log",
"message": "101.28.166.129 - - \"GET /guide.png?v=2 HTTP/1.1\" 200 63133 \"https://ipaper.oeeee.com/ipaper/A/html/2017-02/12/content_6417.htm?from=timeline\u0026isappinstalled=0\u0026wxuid=oq7TJv8NgymKH25j6gniiaODPvfM\u0026wxsalt=731af7\" \"Mozilla/5.0 (iPhone; CPU iPhone OS 10_2 like Mac OS X) AppleWebKit/602.3.12 (KHTML, like Gecko) Mobile/14C92 MicroMessenger/6.5.4 NetType/WIFI Language/zh_CN\"",
"offset": 56286590,
"source": "/data/wwwlogs/test.log",
"type": "log"
}
2017/02/13 09:39:44.899691 output.go:109: DBGoutput worker: publish 2048 events
状态正常
5、安装kibana
解压kibana
elk]# tar -xzf kibana-5.2.0-linux-x86_64.tar.gz -C /usr/local/ 修改监听地址，不然只能本机访问
]# vim /usr/local/kibana-5.2.0-linux-x86_64/config/kibana.ymserver.host: "0.0.0.0" 启动
# /usr/local/kibana-5.2.0-linux-x86_64/bin/kibana & 最后访问测试，正常
https://s5.运维网.com/wyfs02/M02/8D/90/wKioL1iiYT7zdMmDAAD5x7d-HrE602.png-wh_500x0-wm_3-wmp_4-s_2792829748.png
IP访问分布地图
https://s5.运维网.com/wyfs02/M02/8D/94/wKiom1iidQ3S7oblAAQOWAk04p0298.png-wh_500x0-wm_3-wmp_4-s_3395030201.png
页:
[1]