centos7 单节点elk6.2
1、环境要求CPU：建议4核及以上核心
内存 ：建议4G及以上运行内存
硬盘 ：建议200G及以上存储空间
java版本：1.8及以上
# java -version
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
2、需要下载的文件
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.2.tar.gz
https://download.elastic.co/logstash/logstash/logstash-6.2.2.tar.gz
https://artifacts.elastic.co/downloads/kibana/kibana-6.2.2-linux-x86_64.tar.gz
https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.2.2-x86_64.rpm
https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-6.2.2.zip
x-pack试用一年的注册文件需要到官网下载，kibana注册一年之后还需要重新注册。
3、Es配置
解压
# tar -zxf elasticsearch-6.2.2.tar.gz -C /usr/local
# cd /usr/local
创建软链接
# ln -s elasticsearch-6.2.2elasticsearch
新建启动服务账户
# groupadd elastic
# useradd -g elastic -m elastic
# passwd elastic
创建es日志和数据目录，并赋予权限
# mkdir -p /data/usr/elasticsearch/logs
# mkdir /data/usr/elasticsearch/data
# chown -R elastic:elastic /data/usr
设置最大句柄数，不设置启动es会报相应的错误
# vi /etc/sysctl.conf
vm.max_map_count = 655360
配置es内存参数
# vi /usr/local/elasticsearch/config/jvm.options
-Xms2g
-Xmx2g
修改es配置文件
# vi /usr/local/elasticsearch/config/elasticsearch.yml
cluster.name: Micro-Service-Elk
node.name: elkserver
path.data: /data/usr/elasticsearch/data
path.logs: /data/usr/elasticsearch/logs
network.host: 192.168.1.43
http.port: 9200
discovery.zen.ping.unicast.hosts: ["192.168.1.43"]
集群名和节点名皆可自定义
启动es服务
# su elastic
$ nohup /usr/local/elasticsearch/bin/elasticsearch >> /data/usr/elasticsearch/logs/elasticsearch.log&
查看es服务端口
$ netstat -an | grep :9200
tcp6 0 0 192.168.1.43:9200 :::* LISTEN
查看es信息
$ curl -XGET '192.168.1.43:9200/?pretty'
{
"name" : "elkserver",
"cluster_name" : "Micro-Service-Elk",
"cluster_uuid" : "l0WUcZlfSSayO_vTa7D9Uw",
"version" : {
"number" : "6.2.2",
"build_hash" : "10b1edd",
"build_date" : "2018-02-16T19:01:30.685723Z",
"build_snapshot" : false,
"lucene_version" : "7.2.1",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
安装配置x-pack插件
# cp x-pack-6.2.2.zip /data/usr
# chown elastic:elastic /data/usr/x-pack-6.2.2.zip
# su elastic
$ /usr/local/elasticsearch/bin/elasticsearch-plugin install file:///data/usr/x-pack-6.2.2.zip
-> Downloading file:///data/usr/x-pack-6.2.2.zip
[=================================================] 100%
Continue with installation? y
重启es服务
x-pack交互设置es、kibana和logstash_system接入密码
$ cd /usr/local/elasticsearch/bin/x-pack
$ ./setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,kibana,logstash_system.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue y
Enter password for : elastic
Reenter password for : elastic
Enter password for : kibana
Reenter password for : kibana
Enter password for : logstash
Reenter password for : logstash
Changed password for user
Changed password for user
Changed password for user
查看账户信息
$ curl -XGET -u elastic '192.168.1.43:9200/_xpack/security/user?pretty'
Enter host password for user 'elastic':
{
"elastic" : {
"username" : "elastic",
"roles" : [
"superuser"
],
"full_name" : null,
"email" : null,
"metadata" : {
"_reserved" : true
},
"enabled" : true
},
"kibana" : {
"username" : "kibana",
"roles" : [
"kibana_system"
],
"full_name" : null,
"email" : null,
"metadata" : {
"_reserved" : true
},
"enabled" : true
},
"logstash_system" : {
"username" : "logstash_system",
"roles" : [
"logstash_system"
],
"full_name" : null,
"email" : null,
"metadata" : {
"_reserved" : true
},
"enabled" : true
}
}
查看规则信息
$ curl -XGET -u elastic '192.168.1.43:9200/_xpack/security/role'
Enter host password for user 'elastic':
{"kibana_dashboard_only_user":{"cluster":[],"indices":[{"names":[".kibana*"],"privileges":["read","view_index_metadata"]}],"run_as":[],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}},"watcher_admin":{"cluster":["manage_watcher"],"indices":[{"names":[".watches",".triggered_watches",".watcher-history-*"],"privileges":["read"]}],"run_as":[],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}},"logstash_system":{"cluster":["monitor","cluster:admin/xpack/monitoring/bulk"],"indices":[],"run_as":[],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}},"kibana_user":{"cluster":[],"indices":[{"names":[".kibana*"],"privileges":["manage","read","index","delete"]}],"run_as":[],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}},"machine_learning_user":{"cluster":["monitor_ml"],"indices":[{"names":[".ml-anomalies*",".ml-notifications"],"privileges":["view_index_metadata","read"]}],"run_as":[],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}},"remote_monitoring_agent":{"cluster":["manage_index_templates","manage_ingest_pipelines","monitor","cluster:monitor/xpack/watcher/watch/get","cluster:admin/xpack/watcher/watch/put","cluster:admin/xpack/watcher/watch/delete"],"indices":[{"names":[".monitoring-*"],"privileges":["all"]}],"run_as":[],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}},"machine_learning_admin":{"cluster":["manage_ml"],"indices":[{"names":[".ml-*"],"privileges":["view_index_metadata","read"]}],"run_as":[],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}},"watcher_user":{"cluster":["monitor_watcher"],"indices":[{"names":[".watches"],"privileges":["read"]},{"names":[".watcher-history-*"],"privileges":["read"]}],"run_as":[],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}},"monitoring_user":{"cluster":[],"indices":[{"names":[".monitoring-*"],"privileges":["read","read_cross_cluster"]}],"run_as":[],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}},"reporting_user":{"cluster":[],"indices":[],"run_as":[],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}},"kibana_system":{"cluster":["monitor","manage_index_templates","cluster:admin/xpack/monitoring/bulk","manage_saml"],"indices":[{"names":[".kibana*",".reporting-*"],"privileges":["all"]},{"names":[".monitoring-*"],"privileges":["read","read_cross_cluster"]}],"run_as":[],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}},"logstash_admin":{"cluster":[],"indices":[{"names":[".logstash*"],"privileges":["create","delete","index","manage","read"]}],"run_as":[],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}},"transport_client":{"cluster":["transport_client"],"indices":[],"run_as":[],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}},"superuser":{"cluster":["all"],"indices":[{"names":["*"],"privileges":["all"]}],"run_as":["*"],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}},"ingest_admin":{"cluster":["manage_index_templates","manage_pipeline"],"indices":[],"run_as":[],"metadata":{"_reserved":true},"transient_metadata":{"enabled":true}}}
查看license信息
$ curl -XGET -u elastic:elastic "http://192.168.1.43:9200/_license"
{
"license" : {
"status" : "active",
"uid" : "9cd60d1a-1765-4f76-9d68-807951c2ce07",
"type" : "trial",
"issue_date" : "2018-11-09T06:41:05.423Z",
"issue_date_in_millis" : 1541745665423,
"expiry_date" : "2018-12-09T06:41:05.423Z",
"expiry_date_in_millis" : 1544337665423,
"max_nodes" : 1000,
"issued_to" : "Micro-Service-Elk",
"issuer" : "elasticsearch",
"start_date_in_millis" : -1
}
}
4、logstash配置
解压
# tar -xf logstash-6.2.2.tar -C /usr/local
建立软链接
# cd /usr/local
# ln -s logstash-6.2.2 logstash
验证服务
# /usr/local/logstash/bin/logstash -e 'input{stdin{}} output{stdout{}}'
hello elk! （输入）
出现下面输出信息则服务为正常
2018-11-09T09:38:22.062Z elkserver hello elk!
创建logstash日志接入接出配置文件
# cd /usr/local/logstash
# mkdir conf.d
# vi conf.d/logstash_sample.conf
input {
beats {
port => 5044
}
}
output {
elasticsearch {
hosts => ["192.168.1.43:9200"]
index => "%{[@metadata]}-%{[@metadata]}-%{+YYYY.MM.dd}"
user => "elastic"
password => "elastic"
sniffing => true
manage_template => true
document_type => "%{[@metadata]}"
}
}
创建logstash日志和数据目录
# mkdir -p /data/usr/logstash/logs
# mkdir /data/usr/logstash/data
修改logstash配置文件
# vi config/logstash.yml
node.name: logstash_server
path.data: /data/usr/logstash/data
启动logstash服务
# nohup /usr/local/logstash/bin/logstash -f /usr/local/logstash/conf.d/logstash_sample.conf >> /data/usr/logstash/logs/logstash.log &
5、配置kibana
解压
# tar -zxf kibana-6.2.2-linux-x86_64.tar.gz -C /usr/local
创建软链接
# cd /usr/local
# ln -s kibana-6.2.2-linux-x86_64 kibana
修改kibana配置文件
# cd /usr/local/kibana/config
# cp kibana.yml kibana.yml.bak
# vi kibana.yml
# 启动服务端口
server.port: 5601
# 服务ip
server.host: "192.168.1.43"
#服务名称
server.name: "elkserver"
kibana.index: ".kibana"
# es地址
elasticsearch.url: "http://192.168.1.43:9200"
# kibana用户名
elasticsearch.username: "elastic"
# kibana密码
elasticsearch.password: "elastic"
创建kibana日志目录
# mkdir -p /data/usr/kibana/logs
启动kibana服务
# nohup /usr/local/kibana/bin/kibana >>/data/usr/kibana/logs/kibana.log &
kibana安装x-pack插件
# cd /usr/local/kibana/bin
# ./kibana-plugin install file:///data/usr/x-pack-6.2.2.zip
6、filebeat安装配置
安装
# rpm -ivh filebeat-6.2.2-x86_64.rpm
修改filebeat配置文件
# cd /etc/filebeat/
# cp filebeat.yml filebeat.yml.bak
filebeat.prospectors:
- type: log
enabled: true
# 要监控的日志文件地址
- /data/webApps/tomcat/logs/tomcat.log
# 输出的logstash地址
output.logstash:
hosts: ["192.168.1.43:5044"]
测试filebeat
# filebeat -e -c /etc/filebeat/filebeat.yml -d "Publish"
创建filebeat日志目录
# mkdir -p /data/usr/filebeat/logs
启动filebeat服务
# nohup filebeat -e -c /etc/filebeat/filebeat.yml >> /data/usr/filebeat/logs/filebeat.log &
7、配置nginx做kibana反向代理
安装nginx
修改nginx配置
# vi /etc/nginx/nginx.conf
location / {
proxy_pass http://192.168.1.43:5601;
proxy_redirect off;
}
使nginx配置生效
# nginx -s reload
检查
# curl -L http://192.168.1.43:5601
# curl -L http://192.168.1.43
这种是直接把5601端口转到80端口，浏览器输入http://192.168.1.43，即可访问管理界面，也可以下面这种配置，更好一点
# vi /etc/nginx/nginx.conf
location /kibana/ {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header REMOTE-HOST $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_passhttp://192.168.1.43:5601/;
}
# nginx -s reload
修改kibana配置文件
# vi /usr/local/kibana/config/kibana.yml
server.basePath: "/kibana"
这里的配置对应nginx中的配置/kibana/
重启kibana服务
浏览器输入http://192.168.1.43/kibana
登录kibana管理界面，用户和密码是之前kibana配置文件中配置的,elastic elastic
注册license
Management --> Elasticsearch --> License Management--> 选择文件 --> hello-mars-fb832ae0-6be0-4218-8f1c-782f14a85950-v5.json --> Upload --> Confirm
成功，显示
Your license will expire on October 17, 2019 7:59 AM CST.
查看license信息
$ curl -XGET -u elastic:elastic "http://192.168.1.43:9200/_license"
{
"license" : {
"status" : "active",
"uid" : "fb832ae0-6be0-4218-8f1c-782f14a85950",
"type" : "basic",
"issue_date" : "2018-10-16T00:00:00.000Z",
"issue_date_in_millis" : 1539648000000,
"expiry_date" : "2019-10-16T23:59:59.999Z",
"expiry_date_in_millis" : 1571270399999,
"max_nodes" : 100,
"issued_to" : "Hello Mars (Pingan)",
"issuer" : "Web Form",
"start_date_in_millis" : 1539648000000
}
}
可以看到license已经注册完成，到期日为一年后
在kibana管理界面添加索引
Management --> Index Patterns --> Create Index Pattern
输入索引名 filebeat*
接下来点击 Discover 就可以看到日志信息了。
页:
[1]