Awstats流量监控nginx日志全过程

shuaiwei588

　　Awstats流量监控
　　首先下载awstats7.0
　　http://cdnetworks-kr-2.dl.sourceforge.net/project/awstats/AWStats/7.0/awstats-7.0.tar.gz
　　安装，路径选择网站根目录建立awstats
　　我的路径/usr/local/www/php/code/awstats
　　把安装包考到linux下（位置自己定）
　　1．找到awstats-7.0.tar.gz
　　tar –zxvf awstats-7.0.tar.gz
　　#解压安装包
　　2．mv awstats-7.0/usr/local/www/php/code/awstats/awstats
　　#移动到网站根目录
　　3．cd /usr/local/www/php/code/aestats
　　perl awstats_configure.pl
　　perl awstats_configure.pl
　　☆安装实例
　　Perlawstats_configure.pl
　　----- AWStats awstats_configure 1.0 (build 1.9) (c) Laurent Destailleur -----
　　This tool will help you to configure AWStats to analyze statistics for
　　one web server. You can try to use it to let it do all that is possible
　　in AWStats setup, however following the step by step manual setup
　　documentation (docs/index.html) is often a better idea. Above all if:
　　- You are not an administrator user,
　　- You want to analyze downloaded log files without web server,
　　- You want to analyze mail or ftp log files instead of web log files,
　　- You need to analyze load balanced servers log files,
　　- You want to 'understand' all possible ways to use AWStats...
　　Read the AWStats documentation (docs/index.html).
　　-----> Running OS detected: Linux, BSD or Unix
　　Warning: AWStats standard directory on Linux OS is '/usr/local/awstats'.
　　If you want to use standard directory, you should first move all content
　　of AWStats distribution from current directory:
　　/data0/htdocs/winner/awstats
　　to standard directory:
　　/usr/local/awstats
　　And then, run configure.pl from this location.
　　Do you want to continue setup from this NON standard directory [yN] ? y
　　开始安装awstats
　　-----> Check for web server install
　　Enter full config file path of your Web server.
　　Example: /etc/httpd/httpd.conf
　　Example: /usr/local/apache2/conf/httpd.conf
　　Example: c:\Program files\apache group\apache\conf\httpd.conf
　　Config file path ('none' to skip web server setup):
　　> none
　　确定web服务配置文件，因为应用nginx所以不定义
　　Your web server config file(s) could not be found.
　　You will need to setup your web server manually to declare AWStats
　　script as a CGI, if you want to build reports dynamically.
　　See AWStats setup documentation (file docs/index.html)
　　-----> Update model config file '/data0/htdocs/winner/awstats/wwwroot/cgi-bin/awstats.model.conf'
　　File awstats.model.conf updated.
　　-----> Need to create a new config file ?
　　Do you want me to build a new AWStats config/profile
　　file (required if first install) [y/N] ? y
　　建立一个awstats的配置文件
　　-----> Define config file name to create
　　What is the name of your web site or profile analysis ?
　　Example: www.mysite.com
　　Example: demo
　　Your web site, virtual server or profile name:
　　> 113.105.93.6
　　输入你的域名
　　-----> Define config file path
　　In which directory do you plan to store your config file(s) ?
　　Default: /etc/awstats
　　Directory path to store config file(s) (Enter for default):
　　>
　　确定配置文件路径
　　-----> Create config file '/etc/awstats/awstats.113.105.93.6.conf'
　　Config file /etc/awstats/awstats.113.105.93.6.conf created.
　　-----> Add update process inside a scheduler
　　Sorry, configure.pl does not support automatic add to cron yet.
　　You can do it manually by adding the following command to your cron:
　　/data0/htdocs/winner/awstats/wwwroot/cgi-bin/awstats.pl -update -config=113.105.93.6
　　Or if you have several config files and prefer having only one command:
　　/data0/htdocs/winner/awstats/tools/awstats_updateall.pl now
　　Press ENTER to continue...
　　回车
　　A SIMPLE config file has been created: /etc/awstats/awstats.113.105.93.6.conf
　　You should have a look inside to check and change manually main parameters.
　　You can then manually update your statistics for '113.105.93.6' with command:
　　> perl awstats.pl -update -config=113.105.93.6
　　You can also build static report pages for '113.105.93.6' with command:
　　> perl awstats.pl -output=pagetype -config=113.105.93.6
　　Press ENTER to finish...
　　#启动awstats安装向导
　　A．第一个选项选择none因为默认支持apache，nginx选择none（确定配置文件路径）
　　B．然后选Y创建一个新的配置文件
　　C．这里输入自己的域名
　　D．第三个选项是配置文件路径，直接回车使用默认路径
　　E．回车
　　F．回车（完成）
　　4．修改awstats配置文件
　　Vi/etc/awstats/awstat.域名.conf
　　找到LogFile="/var/log/httpd/mylog.log"
　　咱们自己修改成自己的日志位置
　　LogFile="/usr/local/www/weblogs/nginx_%YYYY-24%MM-24%DD-24.log"
　　取到当前日期的前一天的日期
　　5．修改nginx.conf的日志文件格式
　　Vi/usr/local/nginx/conf/nginx.conf
　　log_formatmain'$remote_addr - $remote_user [$time_local] "request" '
　　'$status $body_bytes_sent "$http_referer" '
　　'"$http_user_agent" "$http_x_forwarded_for"';
　　access_log/usr/local/www/weblogs/www.test.com.access.logaccess;
　　6.好，现在差不多了，切割nginx日志
　　编写个sh吧
　　LOGS_PATH=/usr/local/www/weblogs #定义日志路径
　　YESTERDAY=$(date -d "yesterday" +%Y%m%d)取一个昨天的日期
　　mv${LOGS_PATH}/www.purcotton.com.access.log ${LOGS_PATH}/nginx_access/access_${YESTERDAY}.log #移动新日志到指定位置并命名为昨天的日期
　　kill -USR1 $(cat /usr/local/nginx/logs/nginx.pid) #使nginx切换至新日志文件，这个不需要太多的时间瞬间就搞定
　　执行此脚本
　　7.初始化生成静态页面（这个操作会耗时比较大，它会一条一条的去检查这些日志）
　　/安装路径/ awstats/wwwroot/cgi-bin/awstats.pl -update -config=113.105.93.6
　　☆安装实例
　　执行后的输出↓我的log大小在30个G多，具体熬了多久就不知道了，第二天一看就好了估计几个小时吧
　　Create/Update database for config "/etc/awstats/awstats.113.105.93.68.conf" by AWStats version 7.0 (build 1.971)
　　From data in log file "/data1/logs/nginx_access/access_20110425.log"...
　　Phase 1 : First bypass old records, searching new record...
　　Searching new records from beginning of log file...
　　Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Flush history file on disk (unique hosts reach flush limit of 20000)
　　Jumped lines in file: 0
　　Parsed lines in file: 131668109
　　Found 2 dropped records,
　　Found 0 comments,
　　Found 0 blank records,
　　Found 170823 corrupted records,
　　Found 0 old records,
　　Found 131497284 new qualified records.
　　Ok 已经生成
　　8.更新数据的代码
　　/usr/local/www/php/code/liuliang/awstats/tools/awstats_buildstaticpages.pl
　　-update
　　-config=113.105.93.6
　　-dir=/usr/local/www/php/code/liuliang/awstats/log/
　　-lang=cn -awstatsprog=/usr/local/www/php/code/liuliang/awstats/wwwroot/cgi-bin/awstats.pl
　　9.到此通过绝对路径可以访问到，
　　加密方法如下通过apache的htpasswd工具生成一个密钥包
　　这个工具的路径自己查找
　　/usr/bin/htpasswd-cadmin.pass admin
　　生成一个admin.pass的文件
　　修改nginx.conf
　　location ~^/log/ {#重定向好做加密
　　root /usr/local/www/php/code/liuliang/awstats;#定向的目录路径
　　index index.html;#支持的文件类型
　　access_log off;#关掉日志
　　error_log off;
　　auth_basic "admin";#加密
　　auth_basic_user_file /tools/admin.pass;#密钥路径（位置放哪里写哪里）
　　}
　　location ~^/icon/ {#用到的图片路径
　　root /usr/local/www/php/code/liuliang/awstats/wwwroot;#
　　index index.html;
　　access_log off;
　　error_log off;
　　#charser gd2312;
　　}
　　这样，在登录http://www.name.com/log/awstats.www.name.com.html就需要输入密码进行验证了
　　Awstats安装到此结束