|
申明：该文里部分是从别的文章提取，过程属个人部署流程。
Nginx日志分析awstats
Awstats功能及简介： AWStats是在Sourceforge上发展很快的一个基于Perl的WEB日志分析工具。 它可以统计您站点的如下信息： 访问量（UV），访问次数，页面浏览量（PV），点击数，数据流量等 精确到每月、每日、每小时的数据 访问者国家 访问者IP Robots/Spiders的统计 访客持续时间 对不同Files type的统计信息 Pages-URL的统计 访客操作系统浏览器等信息 其它信息（搜索关键字等等）
一、修改nginx日志格式（因为不修改日志格式awstats无法做日志分析）： access_log /usr/local/nginx/logs/access.log; log_format access '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
参数说明： $remote_addr 与$http_x_forwarded_for 用以记录客户端的ip地址； $remote_user ：用来记录客户端用户名称； $time_local ：用来记录访问时间与时区； $request ：用来记录请求的url与http协议； $status ：用来记录请求状态；成功是200， $body_bytes_s ent ：记录发送给客户端文件主体内容大小； $http_referer ：用来记录从那个页面链接访问过来的； $http_user_agent ：记录客户毒啊浏览器的相关信息；
二、nginx日志切割： 脚本: Vim nginx_log.sh #!/bin/bash logs_path="/usr/local/nginx/logs/" #设置日志文件存放目录 pid_path="/usr/local/nginx/logs/nginx.pid" #设置pid文件 mv ${logs_path}access.log ${logs_path}access_$(date -d "yesterday" +"%Y%m%d").log #重命名日志文件，将access.log切割为昨天的日志文件，因为要在每天零点切割，所有的日志格式是前一天的日期，如access_20150916.log kill -USR1 `cat ${pid_path}` #向nginx主进程发信号重新打开日志
三、安装awstats： tar -xvf awstats-7.0.tar.gz mv awstats-7.0 /usr/local/awstats chown -R root:root /usr/local/awstats chmod -R =rwX /usr/local/awstats chmod +x /usr/local/awstats/tools/*.pl chmod +x /usr/local/awstats/wwwroot/cgi-bin/*.pl
执行tools 目录中的awstats_configure.pl 配置向导，创建一个新的统计 cd /usr/local/awstats/tools ./awstats_configure.pl 将会有如下一些提示： ----->Running OS detected: Linux, BSD orUnix
----->Check forweb server install Enterfull config file path of your Web server. Example:/etc/httpd/httpd.conf Example:/usr/local/apache2/conf/httpd.conf Example:c:\Program files\apache group\apache\conf\httpd.conf Config file path ('none'to skip web server setup): >none#这里添none并回车，因为我们没有使用apache 回车之后下一个选项： Yourweb server config file(s) could not be found. Youwill need to setup your web server manually to declare AWStats script as a CGI, ifyou want tobuild reports dynamically. See AWStats setup documentation (file docs/index.html) ----->Update model config file '/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf' File awstats.model.conf updated.
----->Need to create a newconfig file ? Do you want me to build anew AWStats config/profilefile (required if first install) [y/N] ? y #这里选Y，创建一个新的配置文件
----->Define config file name to create What isthe name of your web site or profile analysis ? Example: demo Yourweb site, virtualserver or profile name: >nginx#这里输入你要分析的域名，或是随便一个你易记的配置名并回车
----->Define config file path In which directory do you plan to store your config file(s) ? Default: /etc/awstats Directorypath to store config file(s) (Enter fordefault): > #直接回车，定义你的配置文件存放的路径，使用默认路径/etc/awstats
----->Create config file '/etc/awstats/awstats.nginx.conf' Configfile /etc/awstats/awstats.nginx.conf created. ----->Add updateprocess inside a scheduler Sorry, configure.pl does not support automatic addto cron yet. You can do it manually by adding the following command to yourcron: /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=nginx Or ifyou have several config files and prefer having only onecommand: /usr/local/awstats/tools/awstats_updateall.pl now Press ENTER to continue...#按回车继续 A SIMPLE config file has been created: /etc/awstats/awstats.nginx.conf You should have a lookinside to check and change manually main parameters. Youcan then manually update your statistics for'yuyuanchun.com'withcommand: > perl awstats.pl -update -config=nginx Youcan also build staticreport pages for'nginx'with command:> perl awstats.pl -output=pagetype -config=nginx PressENTER to finish... #回车完成配置文件的创建
默认会生成一个名为awstats.nginx.conf配置文件在/etc/awstats/目录下，修改该配置文件的日志位置 vim /etc/awstats/awstats.nginx.conf LogFile="/usr/local/nginx/logs/access_%YYYY-0%MM-0%DD-24.log"
这里是对应上面Nginx日志切割所生成的目录存放位置，注意awstats的年月日格式，-24表示昨天的日志，-0表示当前的 分析的执行顺序是： Nginx 产生日志 –> 日志切割 –> Nginx 继续产生日志 –> 另存切割日志–> 交由Awstats统计 –> 生成结果
日志说明 # "LogFile" contains the web, ftp or mail server logfile to analyze. #You can also use tags inthisfilename if you need a dynamic file name #depending on date or time (Replacement is made by AWStats at the beginning # ofits execution). This is available tags : # %YYYY-n is replaced with 4 digits year we were n hours ago # %YY-n is replaced with 2 digits year we were n hours ago # %MM-n is replaced with 2 digits month we were n hours ago # %MO-n isreplaced with 3letters month we were n hours ago # %DD-n is replaced with day we were n hours ago # %HH-n isreplaced with hour we were n hours ago # %NS-n isreplaced with number of seconds at 00:00since 1970 # %WM-n is replaced with the week number in month (1-5) # %Wm-n isreplaced with the week number in month (0-4) # %WY-n is replaced with the week number in year (01-52) # %Wy-n is replaced with the week number in year (00-51) # %DW-n is replaced with the day number in week (1-7, 1=sunday) # use n=24if you need (1-7, 1=monday) # %Dw-n isreplaced with the day number inweek (0-6, 0=sunday) # use n=24if you need (0-6, 0=monday) # Use 0for n if you need current year, month, day, hour... n表示时间间隔，我这里就是分析距离当前今天24小时前（昨日）的日志，改成48就是前两天的日志 创建一个awstats用于记录数据的目录 mkdir -p /var/lib/awstats
运行awstats的wwwroot目录中的awstats.pl来测试一下 perl /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=chen
出现以下结果说明正常： Create/Update database for config "/etc/awstats/awstats.chen.conf" by AWStats version 7.3 (build 20140126) From data in log file "/usr/local/nginx/logs/access_20150916.log"... Phase 1 : First bypass old records, searching new record... Direct access after last parsed record (after line 1201) Jumped lines in file: 1201 Found 1201 already parsed records. Parsed lines in file: 0 Found 0 dropped records, Found 0 comments, Found 0 blank records, Found 0 corrupted records, Found 0 old records, Found 0 new qualified records.
Nginx对perl支持并不好，所以在這里用awstats的工具将统计结果生成静态文件：
首先在wwwroot目录下创建一个文件夹： Mkdir -p /home/www/awstats
生成静态文件： perl /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=chen -lang=cn -dir=/home/www/awstats -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl 运行脚本出现以下结果说明正确： Launch update process : "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -update -configdir= Build main page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output Build alldomains page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=alldomains Build allhosts page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=allhosts Build lasthosts page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=lasthosts Build unknownip page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=unknownip Build allrobots page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=allrobots Build lastrobots page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=lastrobots Build session page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=session Build urldetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=urldetail Build urlentry page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=urlentry Build urlexit page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=urlexit Build osdetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=osdetail Build unknownos page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=unknownos Build browserdetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=chen -staticlinks -lang=cn -output=browserdetail
/usr/local/awstats/tools/awstats_buildstaticpages.pl Awstats静态页面生成工具 -update -config=nginx 更新配置项 -lang=cn 语言为中文 -dir=/home/www/awstats 统计结果输出目录 -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl Awstats 日志更新程序路径
修改nginx配置文件，添加日志结果分析站点：
用到apache自带的工具htpasswd： Yum install httpd-tools -y Htpasswd -cd admin.pass admin
修改nginx配置文件： #awstats.conf server { listen 85; server_name localhost; index index.html index.htm index.php default.html default.htm default.php; root /home/www/awstats;
location ~ ^/awstats/ { root /home/www/awstats; index index.html; access_log off; error_log off; charset gb2312; auth_basic "admin"; auth_basic_user_file admin.pass; } location ~ ^/icon/ { root /usr/local/awstats/wwwroot; index index.html; access_log off; error_log off; } autoindex on; access_log off; }
重新加载nginx ln -s /usr/local/nginx/sbin/nginx /usr/bin/ nginx –t #检查nginx配置文件是否有错误 nginx –s reload #加载
添加定时任务：
Vim /usr/local/nginx/awstats.sh
#!/bin/bash /usr/bin/sh /usr/local/nginx/nginx_log.sh /usr/bin/perl /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=chen -lang=cn -dir=/home/www/awstats -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl
Chmod +X /usr/local/nginx/awstats.sh
Crontab -u root -e 01 00 * * * /usr/bin/sh /usr/local/nginx/awstats.sh
http://192.168.1.201:85/awstats.chen.html
| 
QQ群⑧：
窥视卡
雷达卡
发表于 2015-12-17 09:11:46
提升卡
置顶卡
沉默卡
喧嚣卡
变色卡
千斤顶
显身卡