使用Logstash multiline 收集PHP、tomcat等应用服务多行堆栈日志

美奇科技

　　很多时候应用程序出错是都是抛出一堆 堆栈信息（即在日志文件输出多行），此时logstash可以使用multiline的插件收集日志时需要把错误堆栈信息收集为一个记录。multiline字面意思是多行,顾名思义就是对多行日志进行处理。
　　multiline配置与用法
input {
stdin {
codec => multiline {
pattern => "pattern, a regexp"  
negate => "true" or "false"  
what => "previous" or "next"  
}
}
}

　　## pattern支持正则表达式，通过正则表达式匹配日志信息，grok pattern定义的参数也是支持，Eg：%{TIMESTAMP_ISO8601} ，具体信息可以查看grok-patterns
　　## negate  只支持布尔值，true 或者false，默认为false。如果设置为true，表示信息不跟上面正则表达式（pattern）匹配的内容都与匹配的整合，具体整合在前还是在后，看what参数。如果设置为false，即与pattern匹配的内容
　　## what  前一行 或者后一行，指出上面对应的规则与前一行内容收集为一行，还是与后一行整合在一起
　　官方原文

The pattern should match what you believe to be an indicator that the field is part of a multi-line event.
The what must be previous or next and indicates the relation to the multi-line event.
The negate can be true or false (defaults to false). If true, a message not matching the pattern will constitute a match of the multiline filter and the what will be applied. (vice-versa is also true)

　　
　　在这里用PHP-FPM的慢查询日志做个测试
　　PHP-FPM的慢查询日志如下：

[11-Mar-2015 16:54:17]  [pool www] pid 12873
script_filename = /data//index.php
[0x00007f497fa5b620] curl_exec() /data//Account.php:221
[0x00007f497fa5a4e0] call() /data/gintama_app/jidong/game_code/app/controllers/Game.php:31
[0x00007fff29eea180] load() unknown:0
[0x00007f497fa59e18] call_user_func_array() /data/library/BaseCtrl.php:20
[0x00007fff29eea470] handoutAction() unknown:0
[0x00007f497fa59400] run() /data//index.php:30
[11-Mar-2015 16:56:46]  [pool www] pid 12881
script_filename = /data/index.php
[0x00007f497fa5b620] curl_exec() /data//Account.php:221
[0x00007f497fa5a4e0] call() /data/Game.php:31
[0x00007fff29eea180] load() unknown:0
[0x00007f497fa59e18] call_user_func_array() /data/library/BaseCtrl.php:20
[0x00007fff29eea470] handoutAction() unknown:0
[0x00007f497fa59400] run() /data/index.php:30

　　
添加Logstash的配置文件logstash_php-fpm.conf

input {
file {
path => "/tmp/php-slow.log"  ###收集的日志文件路径
codec => multiline {  ###使用multiline
pattern => "^(\[\d{2}-%{MONTH}-\d{4})"  ###使用正则表达式，%{MONTH}是在gork定义好的，这边偷个懒，直接调用
negate => true  ###设置为true，即取正则表达式不匹配的行，然后将内容与上一行或者下一行整合
what => "previous"  ###设置为previous，表示与上一行内容整合在一起。
}
}
}
output{
stdout { codec => rubydebug }
elasticsearch{
hosts => ["110.22.145.155:9200"]
index => "logstash-php_%{+YYYY.MM.dd}"
}
}

　　##测试配置文件是否符合语法
　　{logstash_home}/bin/logstash –t logstash_php-fpm.conf


# /opt/logstash/bin/logstash -t logstash-php_slow.conf
Configuration OK

　　运行logstash，查看输出内容

# /opt/logstash/bin/logstash -f ./logstash-php_slow.conf
Settings: Default pipeline workers: 8
Pipeline main started
{
"@timestamp" => "2017-07-17T05:40:40.310Z",
"message" => "[11-Mar-2015 16:54:17]  [pool www] pid 12873\nscript_filename = /data//index.php\n[0x00007f497fa5b620] curl_exec() /data//Account.php:221\n[0x00007f497fa5a4e0] call() /data/gintama_app/jidong/game_code/app/controllers/Game.php:31\n[0x00007fff29eea180] load() unknown:0\n[0x00007f497fa59e18] call_user_func_array() /data/library/BaseCtrl.php:20\n[0x00007fff29eea470] handoutAction() unknown:0\n[0x00007f497fa59400] run() /data//index.php:30\n  ",
"@version" => "1",
"tags" => [
[0] "multiline"
],
"path" => "/tmp/php-slow.log",
"host" => "test2-web"
}
{
"@timestamp" => "2017-07-17T05:40:47.321Z",
"message" => "[11-Mar-2015 16:56:46]  [pool www] pid 12881\nscript_filename = /data/index.php\n[0x00007f497fa5b620] curl_exec() /data//Account.php:221\n[0x00007f497fa5a4e0] call() /data/Game.php:31\n[0x00007fff29eea180] load() unknown:0\n[0x00007f497fa59e18] call_user_func_array() /data/library/BaseCtrl.php:20\n[0x00007fff29eea470] handoutAction() unknown:0\n[0x00007f497fa59400] run() /data/index.php:30",
"@version" => "1",
"tags" => [
[0] "multiline"
],
"path" => "/tmp/php-slow.log",
"host" => "test2-web"
}

　　Tomcat堆栈信息收集也类似，找出规则，然后进行匹配即可，在此不重复测试