scribe日志系统安装笔记

恒晖瑶

　　版本历史
2011-02-12 17:36:57 完成文章
2011-02-23 10:49:12 更新，修正部分文字
　　目前web访问日志为crontab定时清空，而且负载均衡后，访问随机分配到一台服务器。程序出错后需要分析日志时，需要登录几台机器查看，于是想用一台主机作日志主机，负责收集（按时间顺序而不用sort?）、分析日志。下面的是年前安装scribe日志系统的记录。
一、软件下载
wget http://www.monkey.org/~provos/libevent-1.3.tar.gz
wget https://download.github.com/facebook-scribe-2ee14d3.tar.gz
wget http://nchc.dl.sourceforge.net/project/boost/boost/1.45.0/boost_1_45_0.tar.bz2
wget http://apache.etoak.com//incubator/thrift/0.5.0-incubating/thrift-0.5.0.tar.gz
注：facebook-scribe-2ee14d3.tar.gz为https://github.com/facebook/scribe右上角downloads选择download .tar.gz
二、安装步骤
1～4系统会自带或已安装，未安装的可类似yum install libevent*进行安装。
1、gcc> 3.3.5
gcc -v
which gcc
2、安装ruby
3、安装python
4、安装libevent
5、安装 boost
tar jxvf boost_1_45_0.tar.bz2
cd boost_1_45_0
./bootstrap.sh
./bjam -s HAVE_ICU=1 --prefix=/usr/local/boost --includedir=/usr/local/boost/include --libdir=/usr/local/boost/lib
./bjam install --prefix=/usr/local/boost
6、安装facebook 服务
安装thrift
tar zxvf thrift-0.5.0.tar.gz
cd thrift-0.5.0
./configure --with-boost=/usr/local/boost --with-php-config=/usr/local/php5/bin/php-config
make
make install
安装fb303
cd contrib/fb303
./bootstrap.sh
./configure --with-boost=/usr/local/boost
make
make install
7、安装scribe
环境变量
export BOOST_ROOT=/usr/local/boost
export LD_LIBRARY_PATH=/usr/local/boost/lib::/usr/lob:/usr/local/lib
ldconfig -v
tar zxvf facebook-scribe-2ee14d3.tar.gz
cd facebook-scribe-2ee14d3
./bootstrap.sh
./configure --with-boost=/usr/local/boost --prefix=/usr/local/scribe
make
make install
8、配置测试
mkdir /usr/local/scribe/conf
cp /usr/local/soft/facebook-scribe-2ee14d3/examples/example1.conf /usr/local/scribe/conf
/usr/local/scribe/bin/scribed -c /usr/local/scribe/conf/example1.conf
测试程序：
mkdir  /tmp/scribetest
测试写入：
echo  "hello world"| /usr/local/soft/facebook-scribe-2ee14d3/examples/scribe_cat test
查看结果：
cat  /tmp/scribetest/test/test_current
9、php接口
cd /usr/local/scribe
/usr/local/bin/thrift -o . -I /usr/local/share/ --gen php /usr/local/share/fb303/if/fb303.thrift
/usr/local/bin/thrift -o . -I /usr/local/share/ --gen php /usr/local/soft/facebook-scribe-2ee14d3/if/scribe.thrift
cp /usr/local/soft/thrift-0.5.0/lib/php/src includes -r
mkdir -p includes/packages/fb303
mkdir -p includes/packages/scribe
mv gen-php/fb303/FacebookService.php gen-php/fb303/fb303_types.php includes/packages/fb303/
mv gen-php/scribe/scribe_types.php includes/packages/scribe/
mv gen-php/scribe/scribe.php includes/
rm -rf gen-php
测试程序：
vi test.php
<?php
//参考http://www.ruturaj.net/scribe-php-logging
$GLOBALS['THRIFT_ROOT'] = './includes';
include_once $GLOBALS['THRIFT_ROOT'] . '/scribe.php';
include_once $GLOBALS['THRIFT_ROOT'] . '/transport/TSocket.php';
include_once $GLOBALS['THRIFT_ROOT'] . '/transport/TFramedTransport.php';
include_once $GLOBALS['THRIFT_ROOT'] . '/protocol/TBinaryProtocol.php';
$msg1['category'] = 'keyword';
$msg1['message'] = "This is some message for the category\n";
$msg2['category'] = 'keyword';
$msg2['message'] = "Some other message for the category\n";
//$log_entry = new LogEntry( array('category'=>$category, 'category'=>$category) )
$entry1 = new LogEntry($msg1);
$entry2 = new LogEntry($msg2);
$messages = array($entry1, $entry2);
$socket = new TSocket('localhost', 1463, true);
$transport = new TFramedTransport($socket);
//$protocol = new TBinaryProtocol($trans, $strictRead=false, $strictWrite=true)
$protocol = new TBinaryProtocol($transport, false, false);
//$scribe_client = new scribeClient($iprot=$protocol, $oprot=$protocol)
$scribe_client = new scribeClient($protocol, $protocol);
$transport->open();
$scribe_client->Log($messages);
$transport->close();
?>
执行程序：
/usr/local/php5/bin/php test.php
查看结果：
cat /tmp/scribetest/keyword/keyword_current
三、相关文章
多个服务器日志的排序合并
http://www.chedong.com/blog/archives/001280.html
海量日志分析系统实践
http://wenku.baidu.com/view/a0811f94dd88d0d233d46a6f.html
scribe的安装与使用
http://blogold.chinaunix.net/u3/111447/showart_2164542.html
scribe日志收集器分析
http://blog.iyunv.com/kohaku/archive/2010/12/02/6049183.aspx
使用Scribe来监控大型网络系统
http://blog.iyunv.com/liuzhongbing/archive/2010/06/17/5676096.aspx
nginx-scribe-log
https://github.com/jmj/nginx-scribe-log
Scribe PHP logging
http://www.ruturaj.net/scribe-php-logging
Pipe Apache Logs to Scribe
http://www.silassewell.com/blog/2009/05/12/pipe-apache-or-any-logs-to-scribe