Fluentd+ElasticSearch+Kibana 搭建日誌類文件分析器 安裝篇
一 查看當前機器信息1. 操作系統
$ cd /proc
$ lsb_release -a
LSB Version: :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID: CentOS
Description: CentOS release 6.4 (Final)
Release: 6.4
Codename: Final
2. hostname
$ cat /proc/sys/kernel/hostname
coolbeans-fluentd
$ hostname
coolbeans-fluentd
3. 内存
$ free
total used free shared buffers cached
Mem: 1922464 1067420 855044 0 256628 230204
-/+ buffers/cache: 580588 1341876
Swap: 4128760 0 4128760
$ cat /proc/meminfo
MemTotal: 1922464 kB
MemFree: 854912 kB
Buffers: 256628 kB
Cached: 230204 kB
SwapCached: 0 kB
Active: 690624 kB
Inactive: 193840 kB
Active(anon): 397784 kB
Inactive(anon): 4 kB
Active(file): 292840 kB
Inactive(file): 193836 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 4128760 kB
SwapFree: 4128760 kB
Dirty: 16 kB
Writeback: 0 kB
AnonPages: 397540 kB
Mapped: 26532 kB
Shmem: 164 kB
Slab: 150432 kB
SReclaimable: 91164 kB
SUnreclaim: 59268 kB
KernelStack: 1528 kB
PageTables: 4836 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 5089992 kB
Committed_AS: 645828 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 274520 kB
VmallocChunk: 34359447820 kB
HardwareCorrupted: 0 kB
AnonHugePages: 321536 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 8192 kB
DirectMap2M: 2088960 kB
4. CPU
$ cat /proc/cpuinfo | grep 'model name' |uniq
model name : Intel(R) Xeon(R) CPU X5660@ 2.80GHz
5. Disk
$ df -h
文件系统 容量已用可用 已用%% 挂载点
/dev/mapper/vg_coolbeansfluentd-lv_root
50G3.1G 44G 7% /
tmpfs 939M 0939M 0% /dev/shm
/dev/sda1 485M 64M397M14% /boot
/dev/mapper/vg_coolbeansfluentd-lv_home
439G1.6G415G 1% /home
二 查看Max # of File Descriptors
ulimit -n
三 安裝git
安裝其所依賴之包：
yum install zlib-devel
yum install openssl-devel
yum install perl
yum install cpio
yum install expat-devel
yum install gettext-devel
yum install autoconf
//安裝GIT
wget http://www.codemonkey.org.uk/projects/git-snapshots/git/git-latest.tar.gz
tar xzvf git-latest.tar.gz
cd git-{date}
autoconf
./configure --with-curl=/usr/local
make
make install
四 安裝RUBY
下載ruby-2.0.0-p247.tar.gz
tar -zxvf ruby-2.0.0-p247.tar.gz
cd ruby-2.0.0-p247
./configure-prefix=/opt/modules/ruby
make
make install
添加環境變量：export PATH=/opt/modules/ruby/bin:$PATH
測試RUBY GEM:
$ gem -v
2.0.3
備註：
在安裝RUBY前請確保OPEN-SSL已安裝
五 安裝elasticsearch
下載：elasticsearch-0.90.0.RC2.tar.gz
tar -zxvf elasticsearch-0.90.0.RC2.tar.gz
cd elasticsearch-0.90.0.RC2
啟動： ./bin/elasticsearch -f
六 安裝bundle
$ sudo ./gem install bundle
password for coolbeans:
Fetching: bundler-1.3.5.gem (100%)
Successfully installed bundler-1.3.5
Fetching: bundle-0.0.1.gem (100%)
Successfully installed bundle-0.0.1
Parsing documentation for bundler-1.3.5
Installing ri documentation for bundler-1.3.5
Parsing documentation for bundle-0.0.1
Installing ri documentation for bundle-0.0.1
2 gems installed
七 安裝Kibana
$ cd Kibana-0.2.0
$ bundle install --path vendor/bundle
Fetching gem metadata from http://rubygems.org/.........
Fetching gem metadata from http://rubygems.org/..
Resolving dependencies...
Installing rake (10.1.0)
Installing daemons (1.1.9)
Installing diff-lcs (1.1.3)
Installing eventmachine (1.0.0)
Installing fastercsv (1.5.5)
Installing json (1.7.5)
Installing rack (1.4.1)
Installing rack-protection (1.2.0)
Installing tilt (1.3.3)
Installing sinatra (1.3.3)
Installing thin (1.5.0)
Installing tzinfo (0.3.35)
Using kibana (0.0.1) from source at .
Installing rspec-core (2.11.1)
Installing rspec-expectations (2.11.3)
Installing rspec-mocks (2.11.3)
Installing rspec (2.11.0)
Using bundler (1.3.5)
Your bundle is complete!
It was installed into ./vendor/bundle
啟動Kibana:
bundle exec ruby kibana.rb
默認新動態下，只可在本地訪問：
修改 KibanaConfig.rb16行. 將 127.0.0.1 修改為 0.0.0.0 以綁定所有IP
# The adress ip Kibana should listen on
KibanaHost = '127.0.0.1'
八 安裝Fluented RPM:
執行該腳本即可
http://toolbelt.treasure-data.com/sh/install-redhat.sh
九 安裝ElasticSearch plugin for Fluentd: fluent-plugin-elasticsearch
/usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-elasticsearch
$ sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-elasticsearch
password for coolbeans:
Fetching: fluent-plugin-elasticsearch-0.1.2.gem (100%)
Successfully installed fluent-plugin-elasticsearch-0.1.2
1 gem installed
Installing ri documentation for fluent-plugin-elasticsearch-0.1.2...
Installing RDoc documentation for fluent-plugin-elasticsearch-0.1.2...
修改/etc/td-agent/td-agent.conf為：
<source>
type syslog
port 42185
tag syslog
</source>
<source>
type forward
</source>
<match syslog.**>
type elasticsearch
logstash_format true
flush_interval 10s # for testing
</match>
啟動td-agent
sudo /etc/init.d/td-agent start
十 fluent-plugin-notifier的安裝
學前最新版本為：0.2.1
sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-notifier -v 0.2.1
十一fluent-plugin-mail的安裝
sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-mail
十二 fluent-plugin-tail-ex安裝
在in_tail plugin 的基礎上擴展，其主要新增功能為支持目錄文件收集
安裝命令：sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-tail-ex
十二rsyslogd
系統已自帶
页:
[1]