安装graylog2日志服务器

buhao

　　系统架构：1.最小安装 2.生产部署 3.高可用部署

Highly available
setup with Graylog2 Radio

　　安装前要求：

• 　　Elasticsearch v0.90.10
  
• 　　MongoDB (as recent stable version as possible, at least
  v2.0)
  
• 　　Java 7
  

　　

　　添加系统以外源：
　　http://kernal.blog.运维网.com/8136890/1426095

　　Graylog2 is an excellent centralized logging application created by the excellent guys at torch.sh which utilizes elasticsearchto store logs. It’s scaleable, robust, can deal with a huge number of
logs (if coupled with Graylog2-Radio) and best of all, is open source.
　　There are two components required before we actually install the server
and web component of the app. First, we need to have a mongo db
database.
　　1.[install&configure Mongo]

vim /etc/yum.repos.d/mongodb.repo
[mongodb]
name=MongoDB Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/
enabled=1
gpgcheck=0yum --enablerepo=mongodb -y install mongo-10gen mongo-10gen-server# service mongod start
Starting mongod:                                           [  OK  ]
# chkconfig mongod on　　# 开启服务并加入开机启动
　　我们需要给admin创建一个密码，使用mongo命令
# mongo
MongoDB shell version: 2.6.1
connecting to: test
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
    http://docs.mongodb.org/
Questions? Try the support group
    http://groups.google.com/group/mongodb-user
> use admin                            # 切换到adin数据库
switched to db admin
> db.createUser(
... {
... user : "admin",
... pwd : "logadmin",
... roles : [ "readWrite", "dbAdmin" ]
... }
... )
Successfully added user: { "user" : "admin", "roles" : [ "readWrite", "dbAdmin" ] }
> db.auth('admin','logadmin')
1                                      # 返回1为认证成功　　我们需要为graylog2创建一个数据库：
> use graylog2                         # Mongodb使用use就是创建数据库
switched to db graylog2
> db.createUser(
... {
... user : "graylog",
... pwd : "graylog",
... roles : [ "readWrite", "dbAdmin" ]
... }
... )
Successfully added user: { "user" : "graylog", "roles" : [ "readWrite", "dbAdmin" ] }
> db.auth('graylog', 'graylog')        # 认证测试
1　　2.[install&configure Elasticsearch]
　　运行一个elasticsearch集群不是一个容易的事情，我这里假设只需要一台elasticsearch,其它文档请看http://www.elasticsearch.org/guide/

　　Elasticsearch已经有rpm包所以安装很简单，graylog2仅适用于特定的elasticsearc版本，安装时请注意，elasticsearch需要有java环境。
# java -version
java version "1.7.0_55"
OpenJDK Runtime Environment (rhel-2.4.7.1.el6_5-x86_64 u55-b13)
OpenJDK 64-Bit Server VM (build 24.51-b03, mixed mode)　　如果没有请安装：
# yum -y install java7# wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.10.noarch.rpm# yum -y install elasticsearch-0.90.10.noarch.rpmvim /etc/elasticsearch/elasticsearch.yml
32 cluster.name: graylog2.carson.cn
182 bootstrap.mlockall: true
319 discovery.zen.ping.multicast.enabled: false
324 discovery.zen.ping.unicast.hosts: ["graylog2.carson.cn"]# service elasticsearch start
# service elasticsearch restart
Stopping elasticsearch:                                    [  OK  ]
Starting elasticsearch:                                    [  OK  ]# cat /var/log/elasticsearch/graylog2.carson.cn.log[root@graylog2 elasticsearch]# tail /var/log/elasticsearch/graylog2.carson.log
[2014-06-16 11:06:35,079][INFO ][node                     ] [Spectral] initializing ...
[2014-06-16 11:06:35,088][INFO ][plugins                  ] [Spectral] loaded [], sites []
[2014-06-16 11:06:38,316][INFO ][node                     ] [Spectral] initialized
[2014-06-16 11:06:38,316][INFO ][node                     ] [Spectral] starting ...
[2014-06-16 11:06:38,430][INFO ][transport                ] [Spectral] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.1.186:9300]}
[2014-06-16 11:06:41,488][INFO ][cluster.service          ] [Spectral] new_master [Spectral][6Ry1_DIETdiEtNXloG3K-Q][inet[/192.168.1.186:9300]], reason: zen-disco-join (elected_as_master)
[2014-06-16 11:06:41,605][INFO ][discovery                ] [Spectral] graylog2.carson.cn/6Ry1_DIETdiEtNXloG3K-Q
[2014-06-16 11:06:41,663][INFO ][http                     ] [Spectral] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.1.186:9200]}
[2014-06-16 11:06:41,664][INFO ][node                     ] [Spectral] started
[2014-06-16 11:06:41,695][INFO ][gateway                  ] [Spectral] recovered [0] indices into cluster_state　　能看到这信息，以确保以上配置正确；
　　3.[install&configure graylog2-server]
#wget https://github.com/jaxxstorm/graylog2-server-rpm/releases/download/0.20.0-rc1-1/graylog2-server-0.20.0-rc1.1.el6.noarch.rpm -O graylog2-server-0.20.0-rc1.1.el6.noarch.rpm# yum -y install graylog2-server-0.20.0-rc1.1.el6.noarch.rpm# yum install perl-Digest-SHA　　创建个脚本随机生成字符串：64位
# cat string.sh#!/bin/bash
randstr() {
  index=0
  str=""
  for i in {a..z}; do arr[index]=$i; index=`expr ${index} + 1`; done
  for i in {A..Z}; do arr[index]=$i; index=`expr ${index} + 1`; done
  for i in {0..9}; do arr[index]=$i; index=`expr ${index} + 1`; done
  for i in {1..64}; do str="$str${arr[$RANDOM%$index]}"; done
  echo $str
}
echo `randstr`　　The binaries live in /opt/graylog2 and the config files live in
/etc/graylog2. In this case we need to set a few config options in
/etc/graylog2/server.conf

• 　　is_master = true – you need at least one
  
• 　　password_secret – set a 64 character string here. You’ll need to
  reuse this for any additional server nodes and the web interface portion
  
• 　　root_password_sha2 = enter your root password’s hash here
  
• 　　elasticsearch_shards = 1 – you only have one elasticsearch host at
  the moment, so ensure this is set to 1 (change it if you have more than
  one shard, obviously)
  
• 　　elasticsearch_replicas = 0 – see above
  
• 　　elasticsearch_cluster_name = graylog2 – set this to the same as your elasticsearch cluster name
  
• 　　elasticsearch_transport_tcp_port = 9350 – make sure this is not the same as your elasticsearch node you configured previously
  
• 　　elasticsearch_discovery_zen_ping_multicast_enabled = false
  elasticsearch_discovery_zen_ping_unicast_hosts = localhost:9300 –
  remember what we said about multicast previously? This allows you to
  discover the cluster
  
• 　　Mongodb info – make sure you set useauth to true, and add your database, username and password here
  

　　

cat /etc/graylog2/server.confis_master = true
node_id_file = /etc/graylog2-server-node-id
password_secret = JtMBS4TbbjtPALosVZUk50sUYnsc0pVOkkpKzrD40r6nsoSl5fnSZ6z3PWflFWRy #随机生成
root_password_sha2 = 76cd2c0d...7c1b28bee   # 通过 echo -n yourpassword | shasum -a 256 获得
plugin_dir = plugin
rest_listen_uri = http://127.0.0.1:12900/
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 1                    # 集群数量
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog2
allow_leading_wildcard_searches = false
elasticsearch_cluster_name = graylog2       # 这个名称和elasticsearch.yml中的配置一样
elasticsearch_node_name = graylog2-server
elasticsearch_transport_tcp_port = 9350           # 注意下这个端口号
elasticsearch_discovery_zen_ping_multicast_enabled = false         # 建议使用单播方式
elasticsearch_discovery_zen_ping_unicast_hosts = localhost:9300    # 改成localhost（可以写好多个）
elasticsearch_analyzer = standard
output_batch_size = 5000
processbuffer_processors = 5
outputbuffer_processors = 5
processor_wait_strategy = blocking
ring_size = 1024
mongodb_useauth = true
mongodb_user = grayloguser                # 在Mongodb数据库中创建用户
mongodb_password = gl2-password           # 在Mongodb数据库中创建的密码
mongodb_host = 127.0.0.1                  # Mongodb服务主机ip或者hostname
mongodb_database = graylog2               # 创建的数据库
mongodb_port = 27017                      # 链接数据库的端口号
mongodb_max_connections = 100
mongodb_threads_allowed_to_block_multiplier = 5
transport_email_enabled = false
transport_email_hostname = mail.example.com
transport_email_port = 587
transport_email_use_auth = true
transport_email_use_tls = true
transport_email_use_ssl = true
transport_email_auth_username = you@example.com
transport_email_auth_password = secret
transport_email_subject_prefix = [graylog2]
transport_email_from_email = graylog2@example.cn　　验证以上配置是否正确
# java -jar /opt/graylog2/server/graylog2-server.jar -f /etc/graylog2/server.conf
2014-06-16 12:24:43,890 INFO : org.graylog2.outputs.OutputRegistry - Initialized output .
2014-06-16 12:24:44,023 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Index [graylog2_0] is empty. Not calculating ranges.
2014-06-16 12:24:44,025 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Done calculating index ranges for 1 indices. Took 161ms.
2014-06-16 12:24:44,027 INFO : org.graylog2.system.jobs.SystemJobManager - SystemJob  [org.graylog2.indexer.ranges.RebuildIndexRangesJob] finished in 200ms.
2014-06-16 12:24:49,223 INFO : org.glassfish.jersey.server.ApplicationHandler - Initiating Jersey application, version Jersey: 2.5 2013-12-18 14:27:29...
2014-06-16 12:24:51,970 INFO : org.graylog2.Core - Started REST API at # service graylog2-server start
Starting graylog2-server:
# chkconfig graylog2-server on　　4.[install&configure graylog2-web-interface]
# wget https://github.com/jaxxstorm/graylog2-web-rpm/releases/download/0.20.0-rc1-1/graylog2-web-0.20.0-rc1.1.el6.noarch.rpm -O graylog2-web-0.20.0-rc1.1.el6.noarch.rpm
# yum install graylog2-web-0.20.0-rc1.1.el6.noarch.rpm　　The config file for the web interface is much simpler than the serverinterface. Take a look in /etc/graylog2/web.conf. You need two fields

• 　　graylog2-server.uris – set this to the server address, usually local host unless you made them seperate
  
• 　　application.secret=”" – set this to the same key you have in server.conf password_secret
  

# vim /etc/graylog2/web.confgraylog2-server.uris="
application.secret="JtMBS4TbbjtPALosVZUk50sUYnsc0pVOkkpKzrD40r6nsoSl5fnSZ6z3PWflFWRy"
field_list_limit=100# /opt/graylog2/web/bin/graylog2-web-interface -Dconfig.file=/etc/graylog2/web.conf
Play server process ID is 3153
[info] play - Application started (Prod)
[info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9000　　确保以上配置文件正确！
# service graylog2-web start
# chkconfig graylog2-web on　　先写到这....