nginx日志分析工具awstats
申明：该文里部分是从别的文章提取，过程属个人部署流程。Nginx日志分析awstats
Awstats功能及简介：AWStats是在Sourceforge上发展很快的一个基于Perl的WEB日志分析工具。 它可以统计您站点的如下信息：访问量（UV），访问次数，页面浏览量（PV），点击数，数据流量等精确到每月、每日、每小时的数据访问者国家访问者IPRobots/Spiders的统计访客持续时间对不同Files type的统计信息Pages-URL的统计访客操作系统浏览器等信息其它信息（搜索关键字等等）
一、修改nginx日志格式（因为不修改日志格式awstats无法做日志分析）：access_log /usr/local/nginx/logs/access.log; log_format access '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
参数说明：$remote_addr 与$http_x_forwarded_for 用以记录客户端的ip地址；$remote_user ：用来记录客户端用户名称；$time_local ：用来记录访问时间与时区；$request ：用来记录请求的url与http协议；$status ：用来记录请求状态；成功是200，$body_bytes_s ent ：记录发送给客户端文件主体内容大小；$http_referer ：用来记录从那个页面链接访问过来的；$http_user_agent ：记录客户毒啊浏览器的相关信息；
二、nginx日志切割：脚本:Vimnginx_log.sh#!/bin/bashlogs_path="/usr/local/nginx/logs/" #设置日志文件存放目录pid_path="/usr/local/nginx/logs/nginx.pid" #设置pid文件mv ${logs_path}access.log ${logs_path}access_$(date -d "yesterday" +"%Y%m%d").log#重命名日志文件，将access.log切割为昨天的日志文件，因为要在每天零点切割，所有的日志格式是前一天的日期，如access_20150916.logkill -USR1 `cat ${pid_path}` #向nginx主进程发信号重新打开日志
三、安装awstats：wget http://awstats.sourceforge.net/files/awstats-7.0.tar.gztar -xvf awstats-7.0.tar.gz mv awstats-7.0 /usr/local/awstatschown -R root:root /usr/local/awstatschmod -R =rwX /usr/local/awstatschmod +x /usr/local/awstats/tools/*.pl chmod +x /usr/local/awstats/wwwroot/cgi-bin/*.pl
执行tools 目录中的awstats_configure.pl 配置向导，创建一个新的统计cd /usr/local/awstats/tools./awstats_configure.pl将会有如下一些提示：----->Running OS detected: Linux, BSD orUnix
----->Check forweb server install Enterfull config file path of your Web server. Example:/etc/httpd/httpd.conf Example:/usr/local/apache2/conf/httpd.conf Example:c:\Program files\apache group\apache\conf\httpd.conf Config file path ('none'to skip web server setup): >none#这里添none并回车，因为我们没有使用apache回车之后下一个选项：Yourweb server config file(s) could not be found. Youwill need to setup your web server manually to declare AWStatsscript as a CGI, ifyou want tobuild reports dynamically. See AWStats setup documentation (file docs/index.html) ----->Update model config file '/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf' File awstats.model.conf updated.
----->Need to create a newconfig file ? Do you want me to build anew AWStats config/profilefile (required if first install) ? y #这里选Y，创建一个新的配置文件
----->Define config file name to createWhat isthe name of your web site or profile analysis ?Example: www.mysite.comExample: demo Yourweb site, virtualserver or profile name: >nginx#这里输入你要分析的域名，或是随便一个你易记的配置名并回车
----->Define config file path In which directory do you plan to store your config file(s) ?Default: /etc/awstatsDirectorypath to store config file(s) (Enter fordefault):> #直接回车，定义你的配置文件存放的路径，使用默认路径/etc/awstats
----->Create config file '/etc/awstats/awstats.nginx.conf'Configfile /etc/awstats/awstats.nginx.conf created.----->Add updateprocess inside a scheduler Sorry, configure.pl does not support automatic addto cron yet.You can do it manually by adding the following command to yourcron: /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=nginx Or ifyou have several config files and prefer having only onecommand: /usr/local/awstats/tools/awstats_updateall.pl nowPress ENTER to continue...#按回车继续A SIMPLE config file has been created: /etc/awstats/awstats.nginx.conf You should have a lookinside to check and change manually main parameters. Youcan then manually update your statistics for'yuyuanchun.com'withcommand: > perl awstats.pl -update -config=nginx Youcan also build staticreport pages for'nginx'with command:> perl awstats.pl -output=pagetype -config=nginx PressENTER to finish... #回车完成配置文件的创建
默认会生成一个名为awstats.nginx.conf配置文件在/etc/awstats/目录下，修改该配置文件的日志位置vim /etc/awstats/awstats.nginx.confLogFile="/usr/local/nginx/logs/access_%YYYY-0%MM-0%DD-24.log"
这里是对应上面Nginx日志切割所生成的目录存放位置，注意awstats的年月日格式，-24表示昨天的日志，-0表示当前的分析的执行顺序是：Nginx 产生日志 –> 日志切割 –> Nginx 继续产生日志 –> 另存切割日志–> 交由Awstats统计 –> 生成结果
日志说明# "LogFile" contains the web, ftp or mail server logfile to analyze.#You can also use tags inthisfilename if you need a dynamic file name#depending on date or time (Replacement is made by AWStats at the beginning# ofits execution). This is available tags :# %YYYY-n is replaced with 4 digits year we were n hours ago# %YY-n is replaced with 2 digits year we were n hours ago# %MM-n is replaced with 2 digits month we were n hours ago# %MO-n isreplaced with 3letters month we were n hours ago# %DD-n is replaced with day we were n hours ago# %HH-n isreplaced with hour we were n hours ago# %NS-n isreplaced with number of seconds at 00:00since 1970# %WM-n is replaced with the week number in month (1-5)# %Wm-n isreplaced with the week number in month (0-4)# %WY-n is replaced with the week number in year (01-52)# %Wy-n is replaced with the week number in year (00-51)# %DW-n is replaced with the day number in week (1-7, 1=sunday)# use n=24if you need (1-7, 1=monday)# %Dw-n isreplaced with the day number inweek (0-6, 0=sunday)# use n=24if you need (0-6, 0=monday)# Use 0for n if you need current year, month, day, hour...n表示时间间隔，我这里就是分析距离当前今天24小时前（昨日）的日志，改成48就是前两天的日志创建一个awstats用于记录数据的目录mkdir -p /var/lib/awstats
运行awstats的wwwroot目录中的awstats.pl来测试一下perl /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=chen
出现以下结果说明正常：Create/Update database for config "/etc/awstats/awstats.chen.conf" by AWStats version 7.3 (build 20140126)From data in log file "/usr/local/nginx/logs/access_20150916.log"...Phase 1 : First bypass old records, searching new record...Direct access after last parsed record (after line 1201)Jumped lines in file: 1201 Found 1201 already parsed records.Parsed lines in file: 0 Found 0 dropped records, Found 0 comments, Found 0 blank records, Found 0 corrupted records, Found 0 old records, Found 0 new qualified records.
Nginx对perl支持并不好，所以在這里用awstats的工具将统计结果生成静态文件：
首先在wwwroot目录下创建一个文件夹：Mkdir -p /home/www/awstats
生成静态文件：perl /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=chen-lang=cn -dir=/home/www/awstats -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl运行脚本出现以下结果说明正确：Launch update process : "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -update -configdir=Build main page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -outputBuild alldomains page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=alldomainsBuild allhosts page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=allhostsBuild lasthosts page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=lasthostsBuild unknownip page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=unknownipBuild allrobots page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=allrobotsBuild lastrobots page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=lastrobotsBuild session page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=sessionBuild urldetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=urldetailBuild urlentry page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=urlentryBuild urlexit page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=urlexitBuild osdetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=osdetailBuild unknownos page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=unknownosBuild browserdetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=browserdetail
/usr/local/awstats/tools/awstats_buildstaticpages.pl Awstats静态页面生成工具-update -config=nginx 更新配置项-lang=cn 语言为中文-dir=/home/www/awstats 统计结果输出目录-awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl Awstats 日志更新程序路径
修改nginx配置文件，添加日志结果分析站点：
用到apache自带的工具htpasswd：Yum install httpd-tools -yHtpasswd -cd admin.pass admin
修改nginx配置文件：#awstats.confserver { listen85; server_name localhost; index index.html index.htm index.php default.html default.htm default.php; root /home/www/awstats;
location ~ ^/awstats/ { root /home/www/awstats; index index.html; access_log off; error_log off; charset gb2312; auth_basic"admin"; auth_basic_user_fileadmin.pass; } location ~ ^/icon/ { root /usr/local/awstats/wwwroot; index index.html; access_log off; error_log off; } autoindexon; access_logoff;}
重新加载nginxln -s /usr/local/nginx/sbin/nginx /usr/bin/nginx –t #检查nginx配置文件是否有错误nginx –s reload #加载
添加定时任务：
Vim /usr/local/nginx/awstats.sh
#!/bin/bash/usr/bin/sh /usr/local/nginx/nginx_log.sh/usr/bin/perl /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=chen-lang=cn -dir=/home/www/awstats -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl
Chmod +X /usr/local/nginx/awstats.sh
Crontab -u root -e01 00 * * * /usr/bin/sh /usr/local/nginx/awstats.sh访问：http://192.168.1.201:85就可以看到统计结果静态页面了
http://192.168.1.201:85/awstats.chen.html
页:
[1]