ELK日志平台之ElasticSearch
一、ELKStack简介Elstaicsearch:日志存储和搜索
logstash：日志收集
kibana：日志展示ELK架构示意图：
http://s5.运维网.com/wyfs02/M01/86/77/wKioL1e_uAXhCzMIAABPgVmad_U144.jpg-wh_500x0-wm_3-wmp_4-s_3027742533.jpg
二、ELK安装
环境准备
IP主机名操作系统
192.168.56.11linux-node1centos7
192.168.56.12linux-node2centos71、Elasticsearch安装
安装JDK
# yum install -y java
# java -version
openjdk version "1.8.0_91"
OpenJDK Runtime Environment (build 1.8.0_91-b14)
OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)YUM安装ElasticSearch
（1）下载并安装GPG key
# rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch（2）添加yum仓库
# cat /etc/yum.repos.d/elasticsearch.repo
name=Elasticsearch repository for 2.x packages
baseurl=http://packages.elastic.co/elasticsearch/2.x/centos
gpgcheck=1
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1（3）安装elasticsearch
# yum install -y elasticsearch三、ElasticSearch配置
1、修改配置文件
配置文件在目录/etc/elasticsearch下
# grep '^' elasticsearch.yml
cluster.name: myes #集群名称
node.name: linux-node1 #es所在节点的名称
path.data: /data/es-data #es存放数据的位置
path.logs: /var/log/elasticsearch #es存放日志的位置
bootstrap.mlockall: true #设置内存
network.host: 192.168.56.11 #节点IP
http.port: 9200 #监听端口
#
注：elasticsearch简写成es，es天生支持集群接下来创建数据和日志的目录，并修改属组
mkdir –p /data/es-data
mkdir –p /var/log/elasticsearch
授权
chown –R elasticsearch.elasticsearch /data/es-data
chown –R elasticsearch.elasticsearch /var/log/elasticsearch启动es
# systemctl start elasticsearch
# ps -ef|grep java
logstash 1495 14 10:50 ? 00:01:10 /bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djav.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Djava.io.tmpdir=/var/lib/logstash -Xmx1g -Xss2048k -Djffi.boot.library.path=/opt/logstash/vendor/jruby/lib/jni -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Djava.io.tmpdir=/var/lib/logstash -XX:HeapDumpPath=/opt/logstash/heapdump.hprof -Xbootclasspath/a:/opt/logstash/vendor/jruby/lib/jruby.jar -classpath : -Djruby.home=/opt/logstash/vendor/jruby -Djruby.lib=/opt/logstash/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main --1.9 /opt/logstash/lib/bootstrap/environment.rb logstash/runner.rb agent -f /etc/logstash/conf.d -l /var/log/logstash/logstash.log
**elastic+ 3151 1 93 11:16 ? 00:00:13 /bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -Des.pidfile=/var/run/elasticsearch/elasticsearch.pid -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.conf=/etc/elasticsearch**
root 3243 24860 11:16 pts/0 00:00:00 grep --color=auto java启动后查看进程及端口监听，我在第一次启动时发现进程和端口都没有 查看日志
# tail -n 20 myes.log
elasticsearch hard memlock unlimited
If you are logged in interactively, you will have to re-login for the new limits to take effect.
version, pid, build
initializing ...
modules , plugins , sites
using data paths, mounts [[/ (rootfs)]], net usable_space , net total_space , spins? , types
heap size , compressed ordinary object pointers
max file descriptors for elasticsearch process likely too low, consider increasing to at least
Exception
**java.lang.IllegalStateException: marvel plugin requires the license plugin to be installed**
at org.elasticsearch.marvel.license.LicenseModule.verifyLicensePlugin(LicenseModule.java:37)
at org.elasticsearch.marvel.license.LicenseModule.(LicenseModule.java:25)
at org.elasticsearch.marvel.MarvelPlugin.nodeModules(MarvelPlugin.java:89)
at org.elasticsearch.plugins.PluginsService.nodeModules(PluginsService.java:263)
at org.elasticsearch.node.Node.(Node.java:179)
at org.elasticsearch.node.Node.(Node.java:140)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:143)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:178)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)这种情况下，我们需要安装license
# /usr/share/elasticsearch/bin/plugin install license
-> Installing license...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.3.5/license-2.3.5.zip ...
Downloading .......DONE
Verifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.3.5/license-2.3.5.zip checksums if available ...
Downloading .DONE
Installed license into /usr/share/elasticsearch/plugins/license再次启动ES
systemctl start elasticsearch使用curl命令访问ElasticSearch
# curl http://192.168.56.11:9200
{
"name" : "linux-node1",
"cluster_name" : "myes",
"version" : {
"number" : "2.3.5",
"build_hash" : "90f439ff60a3c0f497f91663701e64ccd01edbb4",
"build_timestamp" : "2016-07-27T10:36:52Z",
"build_snapshot" : false,
"lucene_version" : "5.5.0"
},
"tagline" : "You Know, for Search"
}
#通过web访问es
http://s1.运维网.com/wyfs02/M02/86/77/wKioL1e_uIHwD9j4AABAurvT7eg225.jpg-wh_500x0-wm_3-wmp_4-s_3110555253.jpg
curl加上-i参数后获取 含有协议的头信息
# curl -i 192.168.56.11:9200
HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Content-Length: 311
{
"name" : "linux-node1",
"cluster_name" : "myes",
"version" : {
"number" : "2.3.5",
"build_hash" : "90f439ff60a3c0f497f91663701e64ccd01edbb4",
"build_timestamp" : "2016-07-27T10:36:52Z",
"build_snapshot" : false,
"lucene_version" : "5.5.0"
},
"tagline" : "You Know, for Search"
}还可以进行统计
# curl -i -XGET 'http://192.168.56.11:9200/_count?'
HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Content-Length: 65
{"count":11658,"_shards":{"total":33,"successful":33,"failed":0}}#
#2、插件安装与使用
（1）Marvel插件
Marvel插件：在簇中从每个节点汇集数据。
这个插件必须每个节点都得安装。
Marvel是Elasticsearch的管理和监控工具，在开发环境下免费使用。它包含了一个叫做Sense的交互式控制台，使用户方便的通过浏览器直接与Elasticsearch进行交互。
安装方法：
# /usr/share/elasticsearch/bin/plugin install marvel-agent
-> Installing marvel-agent...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/marvel-agent/2.3.5/marvel-agent-2.3.5.zip ...
Downloading ..........DONE
最后，这个插件安装在
/usr/share/elasticsearch/plugins/marvel-agent（2）head插件
elasticsearch-head是一个elasticsearch的集群管理工具，它是完全由html5编写的独立网页程序，你可以通过插件把它集成到es。
Head插件可以实现基本信息的查看，rest请求的模拟，数据的检索等等。
如果访问elastic官网很慢，我们可以在github上下载安装
# /usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head
-> Installing mobz/elasticsearch-head...
Trying https://github.com/mobz/elasticsearch-head/archive/master.zip ...
插件安装在目录
/usr/share/elasticsearch/plugins/head访问方式
http://192.168.56.11:9200/_plugin/head/http://s5.运维网.com/wyfs02/M00/86/77/wKioL1e_wwrTiCBaAACBFn9OG58461.jpg-wh_500x0-wm_3-wmp_4-s_3394597518.jpg
（3）kopf插件
Kopf是一个ElasticSearch的管理工具，它也提供了对ES集群操作的API。
安装kopf
# /usr/share/elasticsearch/bin/plugin install lmenezes/elasticsearch-kopf
-> Installing lmenezes/elasticsearch-kopf...
Trying https://github.com/lmenezes/elasticsearch-kopf/archive/master.zip ...访问方式
http://192.168.56.11:9200/_plugin/kopf/#!/clusterhttp://s1.运维网.com/wyfs02/M01/86/77/wKioL1e_wxvhaxpQAABTeC7OZ0s943.jpg-wh_500x0-wm_3-wmp_4-s_316360525.jpg
四、ElasticSearch集群搭建
在192.168.56.12上yum安装ElasticSearch，并修改配置文件。
# grep '^' elasticsearch.yml
cluster.name: myes
node.name: linux-node2
path.data: /data/es-data
path.logs: /var/log/elasticsearch
bootstrap.mlockall: true
network.host: 192.168.56.12
http.port: 9200
discovery.zen.ping.unicast.hosts: ["192.168.56.11", "192.168.56.12"]Elasticsearch集群是以组播的形式进行通信的。当node2启动后，它会以组播的形式寻找其他节点，寻找其他cluster.name为myes的节点。在一个集群中，两个节点会进行选举，其中的一个会被选为主节点。主节点负责管理集群的状态。
对于用户而言，可以不用考虑哪个是主节点，连接到其中一个即可。查询时，也不需要主节点转发。
注意 注意 注意 在linux-node2上安装elasticsearch后，我刚开始没有安装license；linux-node2一直加入不到es集群中去，在linux-node1的myes.log中查看发现一直有如下报错：
failed to validate incoming join request from node [{linux-node2}{cZRtReg6R3STovzviXGT6A}{192.168.56.12}{192.168.56.12:9300}]
failed to validate incoming join request from node [{linux-node2}{cZRtReg6R3STovzviXGT6A}{192.168.56.12}{192.168.56.12:9300}]
failed to validate incoming join request from node [{linux-node2}{cZRtReg6R3STovzviXGT6A}{192.168.56.12}{192.168.56.12:9300}]
failed to validate incoming join request from node [{linux-node2}{cZRtReg6R3STovzviXGT6A}{192.168.56.12}{192.168.56.12:9300}]
failed to validate incoming join request from node [{linux-node2}{cZRtReg6R3STovzviXGT6A}{192.168.56.12}{192.168.56.12:9300}]
failed to validate incoming join request from node [{linux-node2}{cZRtReg6R3STovzviXGT6A}{192.168.56.12}{192.168.56.12:9300}]
failed to validate incoming join request from node [{linux-node2}{cZRtReg6R3STovzviXGT6A}{192.168.56.12}{192.168.56.12:9300}]安装license
# /usr/share/elasticsearch/bin/plugin install license
-> Installing license...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.3.5/license-2.3.5.zip ...
Downloading .......DONE
Verifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.3.5/license-2.3.5.zip checksums if available ...
Downloading .DONE
Installed license into /usr/share/elasticsearch/plugins/license
# systemctl restart elasticsearch登陆到http://192.168.56.11:9200/_plugin/head/查看集群状态
可以看到Linux-node2加入到集群中，而且linux-node1是主节点。
绿色是所有分片都处于健康状态。
http://s2.运维网.com/wyfs02/M02/86/78/wKiom1e_wy-yt8K7AACPOYcFeRA045.jpg-wh_500x0-wm_3-wmp_4-s_3886032521.jpg
五、ES监控
我们可以使用curl获取健康数据
# curl -XGET 'http://192.168.56.11:9200/_cluster/health?pretty=true'
{
"cluster_name" : "myes",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 2,
"active_primary_shards" : 39,
"active_shards" : 78,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
#ES中_cat提供了丰富了API，几乎可以监控ES的所有东西
# curl -XGET 'http://192.168.56.11:9200/_cat/health?pretty=true'
1472176491 09:54:51 myes green 2 2 78 39 0 0 0 0 - 100.0%
# curl -XGET 'http://192.168.56.11:9200/_cat/plugins?pretty=true'
linux-node2 license 2.3.5j
linux-node1 head master s /_plugin/head/
linux-node1 kopf 2.0.1s /_plugin/kopf/
linux-node1 license 2.3.5j
linux-node1 marvel-agent 2.3.5j
页:
[1]