Linux 安装 Elasticsearch6.4.x 详细步骤以及问题解决方案
官方文档环境
[*]IP：192.168.1.101
[*]主机名：node1
[*]系统：CentOS 6.8
[*]Java：1.8.0_191
[*]elasticsearch：6.4.2
1 安装java环境
1.1 下载JDK
JDK的下载可以去官网上直接下载，版本为1.8.*即可；
Oracle
1.2 解压jdk
$ mkdir /opt/java
上传jdk包到/opt/java目录下，解压
$ tar xf jdk-8u191-linux-x64.tar.gz
1.3 配置软连接
使jdk指向 jdk1.8.0_191
$ ln -s jdk1.8.0_191 jdk
1.4 配置环境变量
$ vim /etc/profile.d/java.sh
JAVA_HOME=/opt/java/jdk
JRE_HOME=/opt/java/jdk/jre
CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH
PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
使用环境变量生效
$ source /etc/profile
1.5 验证
安装完成后执行如下命令查看安装是否成功；
$ java -version
2 安装ES（elasticsearch）
2.1 新建普通用户
由于Elasticsearch可以接收用户输入的脚本并且执行，为了系统安全考虑，不允许root账号启动，所以建议给Elasticsearch单独创建一个用户来运行Elasticsearch。
$ groupadd elsearch
$ useradd elsearch -g elsearch
$ echo "password" |passwd elsearch --stdin
设置目录属主和属组
$ mkdir -pv /data/ES/
$ chown -R elsearch:elsearch /data/ES/
$ su - elsearch
2.2 下载
ES 去官网直接下载，本人使用的是 6.4.2 版本；因为版本的不同安装head插件的时候安装步骤不同；好像是从5.0 以后的版本安装head 插件的步骤就不一样了；下面会详细介绍。
官网下载路径
点击“MACOS/LINUX”即可，也可以下载rpm包
2.3 解压
将安装包上传到/data/ES/目录下，解压
$ tar xf elasticsearch-6.4.2.tar.gz
$ cd elasticsearch-6.4.2
2.4 修改ES配置文件
没有配置集群，只是简单的配置了一下；详细说明可以参考官网；
$ cp -a config/elasticsearch.yml{,_$(date +%F)}
$ vim config/elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
cluster.name: TRAS-ES # 集群名称
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
node.name: node1 # 主机名
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
path.data: /data/ES/elasticsearch-6.4.2/data # 数据路径
#
# Path to log files:
#
#path.logs: /path/to/logs
path.logs: /data/ES/elasticsearch-6.4.2/logs # 日志路径
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
network.host: 192.168.1.101
#
# Set a custom port for HTTP:
#
#http.port: 9200
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes:
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
创建ES数据文件和日志文件
mkdir -p /data/ES/elasticsearch-6.4.2/data
mkdir -p /data/ES/elasticsearch-6.4.2/logs
2.5 启动
执行ES文件，进入到bin 目录下执行 ./elasticsearch 命令就可以了，执行 ./elasticesrarch -d 是后台运行
$ ./bin/elasticsearch
initializing ...
using data paths, mounts [], net usable_space , net total_space , types
heap size , compressed ordinary object pointers
node name , node ID
version, pid, build, OS, JVM
JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.Ol6oSUD9, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/data/ES/elasticsearch-6.4.2, -Des.path.conf=/data/ES/elasticsearch-6.4.2/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
loaded module
no plugins loaded
parsed roles from file
controller (64 bit): Version 6.4.2 (Build 660eefe6f2ea55) Copyright (c) 2018 Elasticsearch BV
Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
using discovery type
initialized
starting ...
publish_address {192.168.1.101:9300}, bound_addresses {192.168.1.101:9300}
bound or publishing to a non-loopback address, enforcing bootstrap checks
zen-disco-elected-as-master ( nodes joined)[, ], reason: new_master {node1}{IiSDNdfAS_WfGNeRQWX-Rg}{lVUD4owbQ42oQ9czQOQ_Tw}{192.168.1.101}{192.168.1.101:9300}{ml.machine_memory=16604286976, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
new_master {node1}{IiSDNdfAS_WfGNeRQWX-Rg}{lVUD4owbQ42oQ9czQOQ_Tw}{192.168.1.101}{192.168.1.101:9300}{ml.machine_memory=16604286976, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master source nodes joined)[, ]]])
publish_address {192.168.1.101:9200}, bound_addresses {192.168.1.101:9200}
started
Failed to clear cache for realms [[]]
license mode - valid
recovered indices into cluster_state
2.6 启动遇到的问题：
2.6.1 max file descriptors for elasticsearch process is too low, increase to at least
max file descriptors for elasticsearch process is too low, increase to at least
原因：无法创建本地文件问题,用户最大可创建文件数太小
解决方案：切到root 用户下
$ vim /etc/security/limits.conf 在文件的末尾添加下面的参数值：
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096
前面的*符号必须带上，然后重新启动就可以了。执行完成后可以使用命令 ulimit -n 查看进程数
2.6.2 max virtual memory areas vm.max_map_count is too low, increase to at least
ERROR: bootstrap checks failed
: max virtual memory areas vm.max_map_count is too low, increase to at least
... ...
Native controller process has stopped - no new native processes can be started
原因：最大虚拟内存太小,需要修改系统变量的最大值。
解决方案：切换到root用户,修改配置sysctl.conf增加配置值： vm.max_map_count=262144
$ vim /etc/sysctl.conf
vm.max_map_count=262144
执行命令
sysctl -p
2.6.3 max number of threads for user likely too low, increase to at least
原因：无法创建本地线程问题,用户最大可创建线程数太小
解决方案：切换到root用户，进入limits.d目录下，修改90-nproc.conf 配置文件。
vi /etc/security/limits.d/90-nproc.conf
找到如下内容：
* soft nproc 1024
#修改为
* soft nproc 2048
2.6.4 system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
原因：Centos6不支持SecComp，而ES6.4.2默认bootstrap.system_call_filter为true进行检测，所以导致检测失败，失败后直接导致ES不能启动。
详见 ：https://github.com/elastic/elasticsearch/issues/22899
解决方法：在elasticsearch.yml中新增配置bootstrap.system_call_filter，设为false，注意要在Memory下面:
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
然后重新启动ES服务 就可以了。
2.7 验证
执行curl 'http://自己配置的IP地址:9200/' 命令，就出现下面的结果
$ curl 'http://192.168.1.101:9200/'
{
"name" : "node1",
"cluster_name" : "TRAS-ES",
"cluster_uuid" : "g_YrGeWpT4aEMKAdigk-rQ",
"version" : {
"number" : "6.4.2",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "04711c2",
"build_date" : "2018-09-26T13:34:09.098244Z",
"build_snapshot" : false,
"lucene_version" : "7.4.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
如果没有什么问题话，就可以停掉前台启动，使用后台启动ES；
$ ./bin/elasticsearch -d
$ ss -tnlp |grep 9200
LISTEN 0 128 ::ffff:192.168.1.101:9200 :::* users:(("java",6686,246))
3 安装elasticsearch-head 插件
为了便于管理ES，本文使用head插件，这是最初级的管理工具，在浏览器中显示ES集群，索引等信息，十分好用。
3.1 安装node.js：
grunt是基于Node.js的项目构建工具，下面先安装node.js，
3.1.1 配置nodejs的yum源,执行curl -sL https://rpm.nodesource.com/setup_8.x | bash -，显示如下：
$ curl -sL https://rpm.nodesource.com/setup_8.x | bash -
## Installing the NodeSource Node.js 8.x LTS Carbon repo...
## Inspecting system...
+ rpm -q --whatprovides redhat-release || rpm -q --whatprovides centos-release || rpm -q --whatprovides cloudlinux-release || rpm -q --whatprovides sl-release
+ uname -m
## Confirming "el6-x86_64" is supported...
+ curl -sLf -o /dev/null 'https://rpm.nodesource.com/pub_8.x/el/6/x86_64/nodesource-release-el6-1.noarch.rpm'
## Downloading release setup RPM...
+ mktemp
+ curl -sL -o '/tmp/tmp.UGP0HdCahJ' 'https://rpm.nodesource.com/pub_8.x/el/6/x86_64/nodesource-release-el6-1.noarch.rpm'
## Installing release setup RPM...
+ rpm -i --nosignature --force '/tmp/tmp.UGP0HdCahJ'
## Cleaning up...
+ rm -f '/tmp/tmp.UGP0HdCahJ'
## Checking for existing installations...
+ rpm -qa 'node|npm' | grep -v nodesource
## Run `sudo yum install -y nodejs` to install Node.js 8.x LTS Carbon and npm.
## You may also need development tools to build native addons:
sudo yum install gcc-c++ make
## To install the Yarn package manager, run:
curl -sL https://dl.yarnpkg.com/rpm/yarn.repo | sudo tee /etc/yum.repos.d/yarn.repo
sudo yum install yarn
3.1.2 安装nodejs
$ yum install -y nodejs
版本：2:8.12.0-1nodesource
3.1.3 验证
可以使用命令 node -v 验证是否安装成功，同时npm 也安装成功了；执行命令 npm -v 也是可以验证的。
$ node -v
v8.12.0
$ npm -v
6.4.1
3.2 下载head安装包
下载地址：https://github.com/mobz/elasticsearch-head/archive/master.zip
$ cd /data/ES
$ wget https://github.com/mobz/elasticsearch-head/archive/master.zip
$ unzip master.zip
$ mv elasticsearch-head-master elasticsearch-head
由于head 插件不能放在elasticsearch-6.4.2 文件夹里，head 插件需要单独放，单独去执行；
所以在elasticsearch-6.4.2 同级目录下解压了 head 插件；解压出来的文件名字，
3.3 安装grunt
由于head 插件的执行文件是有grunt 命令来执行的，所以这个命令必须安装
$ cd /data/ES/elasticsearch-head
$ sudo npm install grunt --save-dev
npm WARN deprecated coffee-script@1.10.0: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN elasticsearch-head@0.0.0 license should be a valid SPDX license expression
+ grunt@1.0.1
added 94 packages from 61 contributors and audited 162 packages in 11.256s
found 2 low severity vulnerabilities
run `npm audit fix` to fix them, or `npm audit` for details
# 安装
$ sudo npm install
npm WARN deprecated http2@3.3.7: Use the built-in module in node 9.0.0 or newer, instead
> phantomjs-prebuilt@2.1.16 install /data/ES/elasticsearch-head/node_modules/phantomjs-prebuilt
> node install.js
PhantomJS not found on PATH
Downloading https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2
Saving to /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
Receiving...
[========================================] 100%
Received 22866K total.
Extracting tar contents (via spawned process)
Removing /data/ES/elasticsearch-head/node_modules/phantomjs-prebuilt/lib/phantom
Copying extracted folder /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2-extract-1540884540095/phantomjs-2.1.1-linux-x86_64 -> /data/ES/elasticsearch-head/node_modules/phantomjs-prebuilt/lib/phantom
Writing location.js file
Done. Phantomjs binary available at /data/ES/elasticsearch-head/node_modules/phantomjs-prebuilt/lib/phantom/bin/phantomjs
npm WARN elasticsearch-head@0.0.0 license should be a valid SPDX license expression
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.4 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
added 413 packages from 567 contributors and audited 1750 packages in 1054.629s
found 23 vulnerabilities (17 low, 6 high)
run `npm audit fix` to fix them, or `npm audit` for details
# 修复漏洞
$ sudo npm audit fix
npm WARN elasticsearch-head@0.0.0 license should be a valid SPDX license expression
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.4 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
+ grunt-contrib-watch@1.1.0
+ grunt@1.0.3
added 19 packages from 14 contributors, removed 5 packages and updated 9 packages in 6.614s
fixed 5 of 23 vulnerabilities in 1750 scanned packages
3 package updates for 18 vulns involved breaking changes
(use `npm audit fix --force` to install breaking changes; or refer to `npm audit` for steps to fix these manually)
3.4 检查是否安装成功
$ grunt -version
grunt-cli v1.3.1
grunt v1.0.1
3.5 修改head插件配置
修改配置文件，cd 进入elasticsearch-head 文件夹下，执行命令vim Gruntfile.js文件：增加hostname属性，设置为*；
$ pwd
/data/ES/elasticsearch-head
$ vim Gruntfile.js
... ...
77 watch: {
78 "scripts": {
79 files: ['src/**/*', 'test/spec/*' ],
80 tasks: ['default'],
81 options: {
82 spawn: false
83 }
84 },
85 "grunt": {
86 files: [ 'Gruntfile.js' ]
87 }
88 },
89
90 connect: {
91 server: {
92 options: {
93 port: 9100,
94 hostname: '*', # 添加此行
95 base: '.',
96 keepalive: true
97 }
98 }
99 }
100
101 });
102
... ...
修改 vim _site/app.js 文件：修改head的连接地址:
$ pwd
/data/ES/elasticsearch-head
$ vim _site/app.js
把
4354 this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://localhost:9200";
改为：
4354 this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://192.168.1.101:9200";
3.6 运行elasticsearch-head
前台运行
$ cd /data/ES/elasticsearch-head
$ grunt server
(node:13043) ExperimentalWarning: The http2 module is an experimental API.
Running "connect:server" (connect) task
Waiting forever...
Started connect web server on http://localhost:9100
后台运行
$ cd /data/ES/elasticsearch-head
$ grunt server &
# 查看是否后台运行
$ ps -ef|grep grunt
elsearch 17512 306331 17:26 pts/1 00:00:00 grunt
elsearch 17583 306330 17:26 pts/1 00:00:00 grep --color=auto grunt
执行完成后就OK了
3.7 web访问elasticsearch-head
URL：http://192.168.1.101:9100/
http://i2.运维网.com/images/blog/201810/30/760d93d15b85b5f238afa7e25447e8f6.jpg
看到上面的出现的健康值了吗，说明的连接还是有问题的
在elasticsearch中启用CORS
$ vim elasticsearch-6.4.2/config/elasticsearch.yml
... ...
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
# 追加
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-origin: "*"是有效值，但它被视为安全风险，因为您的群集可以从任何地方开放交叉。
然后重新执行ES./elasticsearch 成功起来就可以了，执行结果就是这样的
http://i2.运维网.com/images/blog/201810/30/5fed27ffa698dadf60a81388afbe3c95.jpg
3.8 说明
elasticsearch：
URL：http://192.168.1.101:9200/
版本：6.4.2
安装路径：/data/ES
elasticsearch-head:
http://192.168.1.101:9100/
4 分词安装：
4.1 下载
4.1.1 方法一
下载对应版本的ik包：
https://github.com/medcl/elasticsearch-analysis-ik/releases
在/data/ES/elasticsearch-6.4.2/中新建plugins/ik目录
$ cd /data/ES/elasticsearch-6.4.2/
$ mkdir -pv plugins/ik
上传elasticsearch-analysis-ik-6.4.2.zip到当前目录下
解压到plugins/ik
$ unzip elasticsearch-analysis-ik-6.4.2.zip -d plugins/ik
4.1.2 方法二：
$ cd /data/ES/elasticsearch-6.4.2/
$./bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.4.2/elasticsearch-analysis-ik-6.4.2.zip
注意：替换6.4.2为你的elasticsearch版本
4.2 重启elasticsearch服务
首先杀死elasticsearch服务，然后启动
./bin/elasticsearch -d
4.3 快速示例
4.3.1 创建一个索引
curl -XPUT http://http://192.168.1.101:9200/index
4.3.2 创建一个映射
curl -XPOST http://192.168.1.101:9200/index/fulltext/_mapping -H 'Content-Type:application/json' -d'
{
"properties": {
"content": {
"type": "text",
"analyzer": "ik_max_word",
"search_analyzer": "ik_max_word"
}
}
}'
4.3.3 索引一些文档
curl -XPOST http://192.168.1.101:9200/index/fulltext/1 -H 'Content-Type:application/json' -d'
{"content":"美国留给伊拉克的是个烂摊子吗"}
'
curl -XPOST http://192.168.1.101:9200/index/fulltext/2 -H 'Content-Type:application/json' -d'
{"content":"公安部：各地校车将享最高路权"}
'
curl -XPOST http://192.168.1.101:9200/index/fulltext/3 -H 'Content-Type:application/json' -d'
{"content":"中韩渔警冲突调查：韩警平均每天扣1艘中国渔船"}
'
curl -XPOST http://192.168.1.101:9200/index/fulltext/4 -H 'Content-Type:application/json' -d'
{"content":"中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"}
'
4.3.4 查询显示
curl -XPOST http://192.168.1.101:9200/index/fulltext/_search?pretty-H 'Content-Type:application/json' -d'
{
"query" : { "match" : { "content" : "中国" }},
"highlight" : {
"pre_tags" : ["", ""],
"post_tags" : ["", ""],
"fields" : {
"content" : {}
}
}
}
'
结果:
{
"took": 14,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 2,
"hits": [
{
"_index": "index",
"_type": "fulltext",
"_id": "4",
"_score": 2,
"_source": {
"content": "中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"
},
"highlight": {
"content": [
"中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首 "
]
}
},
{
"_index": "index",
"_type": "fulltext",
"_id": "3",
"_score": 2,
"_source": {
"content": "中韩渔警冲突调查：韩警平均每天扣1艘中国渔船"
},
"highlight": {
"content": [
"均每天扣1艘中国渔船 "
]
}
}
]
}
}
4.4 web重新访问elasticsearch-head
URL：http://192.168.1.101:9100/
http://i2.运维网.com/images/blog/201810/30/befbdea99946e1c0444d793797f3ce53.jpg
[*]注：集群健康值上含有黄底纹的的颜色说明未进行集群设置。
刚才测试的数据已添加：
http://i2.运维网.com/images/blog/201810/30/3e843480e1fa58927b8e7c848b9872c3.jpg
5 参考
http://blog.运维网.com/4925054/2083939
页:
[1]