LAMP搭建15：Apache禁止指定user_agent访问
查看Apache访问日志的user_agent：# cd logs/
# ls
access_logtest.com-access_20170114_logtest.com-error_log
error_log test.com-access_20170115_log
httpd.pid test.com-access_log
#
# ls
access_logtest.com-access_20170114_logtest.com-error_log
error_log test.com-access_20170115_log
httpd.pid test.com-access_log
# tail test.com-access_20170115_log
……
192.168.147.1 - - "GET /data/info.php HTTP/1.1" 403 215 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
192.168.147.1 - - "GET /favicon.ico HTTP/1.1" 200 5558 "http://www.test.com/data/info.php" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
127.0.0.1 - - "GET http://www.test.com/data/info.php HTTP/1.1" 200 20 "-" "curl/7.19.7 (i386-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2"
可以禁止指定user_agent访问我们的网站，尤其是浏览器爬虫偷跑流量。编辑虚拟主机配置文件：
# vim /usr/local/apache2/conf/extra/httpd-vhosts.conf
加入如下内容：禁止curl和chrome
……
RewriteEngine on
RewriteCond %{HTTP_HOST} ^www.aaa.com$
RewriteCond %{HTTP_HOST} ^www.bbb.com$
RewriteRule ^/(.*)$ http://www.test.com/$1
RewriteCond %{HTTP_USER_AGENT} ^.*curl.*
RewriteCond %{HTTP_USER_AGENT} ^.*chrome*
RewriteRule .* -
AllowOverride AuthConfig
AuthName "username"
AuthType Basic
……
# apachectl -t
Syntax OK
# apachectl graceful
禁用后chrome浏览器不再能访问我们的网站，
但是其他浏览器仍可以访问：IE浏览器可以访问
360浏览器也可以访问
curl测试返回值也都是403，
# curl -x127.0.0.1:80 www.test.com
403 Forbidden
Forbidden
You don't have permission to access /
on this server.
# curl -x127.0.0.1:80 www.test.com/forum.php
403 Forbidden
Forbidden
You don't have permission to access /forum.php
on this server.
# curl -x127.0.0.1:80 www.test.com/data/info.php
403 Forbidden
Forbidden
You don't have permission to access /data/info.php
on this server.
#
正常情况下的curl能找到的页面返回200，找不到的页面返回404，禁止curl之后都返回403。这里我们还需要使用curl进行测试，先去掉curl的禁用：
# vim /usr/local/apache2/conf/extra/httpd-vhosts.conf
……
RewriteEngine on
RewriteCond %{HTTP_HOST} ^www.aaa.com$
RewriteCond %{HTTP_HOST} ^www.bbb.com$
RewriteRule ^/(.*)$ http://www.test.com/$1
#RewriteCond %{HTTP_USER_AGENT} ^.*curl.*
RewriteCond %{HTTP_USER_AGENT} ^.*chrome*
RewriteRule .* -
……
# apachectl -t
Syntax OK
# apachectl graceful
# curl -x192.168.147.132:80 www.test.com/forum.php -I
HTTP/1.1 200 OK
Date: Sat, 14 Jan 2017 19:38:46 GMT
Server: Apache/2.2.9 (Unix) PHP/5.4.36
X-Powered-By: PHP/5.4.36
Set-Cookie: sTi8_2132_saltkey=QmAybXxa; expires=Mon, 13-Feb-2017 19:38:46 GMT; path=/; httponly
Set-Cookie: sTi8_2132_lastvisit=1484419126; expires=Mon, 13-Feb-2017 19:38:46 GMT; path=/
Set-Cookie: sTi8_2132_sid=MVss2L; expires=Sun, 15-Jan-2017 19:38:46 GMT; path=/
Set-Cookie: sTi8_2132_checkpatch=1; expires=Sat, 14-Jan-2017 19:39:46 GMT; path=/
Set-Cookie: sTi8_2132_lastact=1484422726%09forum.php%09; expires=Sun, 15-Jan-2017 19:38:46 GMT; path=/
Set-Cookie: sTi8_2132_onlineusernum=1; expires=Sat, 14-Jan-2017 19:43:46 GMT; path=/
Set-Cookie: sTi8_2132_sid=MVss2L; expires=Sun, 15-Jan-2017 19:38:46 GMT; path=/
Cache-Control: max-age=0
Expires: Sat, 14 Jan 2017 19:38:46 GMT
Content-Type: text/html; charset=gbk
使用-A选项模拟user_agent，不含"chrome"时可以访问，加入"chrome"后不能访问
# curl -A "asdfghjkl" -x192.168.147.132:80 www.test.com/forum.php -I
HTTP/1.1 200 OK
Date: Sat, 14 Jan 2017 19:40:17 GMT
Server: Apache/2.2.9 (Unix) PHP/5.4.36
X-Powered-By: PHP/5.4.36
Set-Cookie: sTi8_2132_saltkey=ZggZv4O6; expires=Mon, 13-Feb-2017 19:40:17 GMT; path=/; httponly
Set-Cookie: sTi8_2132_lastvisit=1484419217; expires=Mon, 13-Feb-2017 19:40:17 GMT; path=/
Set-Cookie: sTi8_2132_sid=RG77fb; expires=Sun, 15-Jan-2017 19:40:17 GMT; path=/
Set-Cookie: sTi8_2132_lastact=1484422817%09forum.php%09; expires=Sun, 15-Jan-2017 19:40:17 GMT; path=/
Set-Cookie: sTi8_2132_onlineusernum=1; expires=Sat, 14-Jan-2017 19:45:17 GMT; path=/
Set-Cookie: sTi8_2132_sid=RG77fb; expires=Sun, 15-Jan-2017 19:40:17 GMT; path=/
Cache-Control: max-age=0
Expires: Sat, 14 Jan 2017 19:40:17 GMT
Content-Type: text/html; charset=gbk
# curl -A "asdchromefghjkl" -x192.168.147.132:80 www.test.com/forum.php -I
HTTP/1.1 403 Forbidden
Date: Sat, 14 Jan 2017 19:41:33 GMT
Server: Apache/2.2.9 (Unix) PHP/5.4.36
Content-Type: text/html; charset=iso-8859-1
页:
[1]