问题一
[2016-11-07T11:37:54,191][WARN ][o.e.b.JNANatives ] Unable to lock JVM Memory: error=12, reason=Cannot allocate memory
[2016-11-07T11:37:54,191][WARN ][o.e.b.JNANatives ] This can result in part of the JVM being swapped out.
[2016-11-07T11:37:54,191][WARN ][o.e.b.JNANatives ] Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536
[2016-11-07T11:37:54,191][WARN ][o.e.b.JNANatives ] These can be adjusted by modifying /etc/security/limits.conf, for example:
# allow user 'elasticsearch' mlockall
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
解决方法:
/etc/security/limits.conf
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
问题二
Found index level settings on node level configuration.
Since elasticsearch 5.x index level settings can NOT be set on the nodes
configuration like the elasticsearch.yaml, in system properties or command line
arguments.In order to upgrade all indices the settings must be updated via the
/${index}/_settings API. Unless all settings are dynamic all indices must be closed
in order to apply the upgradeIndices created in the future should use index templates
to set default values.
问题三
java.lang.IllegalArgumentException: unknown setting [node.rack] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
问题四
[2016-11-07T12:26:05,957][ERROR][o.e.b.Bootstrap ] Exception
java.lang.IllegalArgumentException: unknown setting [script.indexed] did you mean any of [script.inline, script.ingest]?
问题五,
max file descriptors [65535] for elasticsearch process likely too low, increase to at least [65536]
max number of threads [1024] for user [elasticsearch] likely too low, increase to at least [2048]
解决方法:
报错信息直接提示了修改limits.conf的相关参数
elasticsearch soft nofile 65536
elasticsearch hard nofile 131072
elasticsearch soft nproc 2048
elasticsearch hard nproc 4096
5,优化elasticsearch配置文件
grep 'Xm' /etc/elasticsearch/jvm.options
## -Xms4g
## -Xmx4g
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms20g
-Xmx20g
6,logstash安装文件位置有变动
/usr/share/logstash/
启动
bin/logstash --path.settings /etc/logstash/logstash.yml
配置logstash-input-kafka时报错
[2016-11-11T11:17:35,740][ERROR][logstash.inputs.kafka ] Unknown setting 'zk_connect' for kafka
[2016-11-11T11:17:35,742][ERROR][logstash.inputs.kafka ] Unknown setting 'topic_id' for kafka
[2016-11-11T11:17:35,742][ERROR][logstash.inputs.kafka ] Unknown setting 'reset_beginning' for kafka
检查语法没有错误
vim /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-5.0.5/DEVELOPER.md
进排查发现是缺少依赖
* Apache Kafka version 0.8.1.1
* jruby-kafka library(这个缺乏导致,坑爹rpm安装的logstash5.0竟然缺东西)