python找寻合适的日志库logging Handler——Handler自定义实现

kaywang

最近在用python tornado开发一个app的服务端。投产的系统肯定需要包含日志功能，这里就自然想到了用python自带的logging库。



logging中日志内容的输出都交由Handler来实现，但是logging中的自带的Handler都不能满足我们的需求。

我们希望能按时间段分割日志，如果使用FileHandler日志只能保存在一个文件，到后期日志文件会非常大，读写都成问题；而TimedRotatingFileHandler虽然可分割日志但是多进程时可能会造成日志文件被相互覆盖，导致日志丢失。

如此我便开始踏上找寻合适Handler的路上。



首先尝试使用FileHandler，然后写个脚本（比如用supervisord）定时切分日志的方式。但这里有一个问题是日志文件会被正在运行FileHandler保持，无法重命名，日志还是会一直写到同一个文件，尝试失败。



然后尝试使用继承logging自带的切分文件处理TimedRotatingFileHandler再重写处理切分日志的方法。

这里使用了一个网友所写的类






#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# Copyright 2012 Ethan Zhang
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

import time
import os
from logging.handlers import TimedRotatingFileHandler
class MultiProcessTimedRotatingFileHandler(TimedRotatingFileHandler):
def doRollover(self):
"""
do a rollover; in this case, a date/time stamp is appended to the filename
when the rollover happens.  However, you want the file to be named for the
start of the interval, not the current time.  If there is a backup count,
then we have to get a list of matching filenames, sort them and remove
the one with the oldest suffix.
"""
if self.stream:
self.stream.close()
# get the time that this sequence started at and make it a TimeTuple
t = self.rolloverAt - self.interval
if self.utc:
timeTuple = time.gmtime(t)
else:
timeTuple = time.localtime(t)
dfn = self.baseFilename + "." + time.strftime(self.suffix, timeTuple)
#if os.path.exists(dfn):
#    os.remove(dfn)
if not os.path.exists(dfn):
os.rename(self.baseFilename, dfn)
if self.backupCount > 0:
# find the oldest log file and delete it
#s = glob.glob(self.baseFilename + ".20*")
#if len(s) > self.backupCount:
#    s.sort()
#    os.remove(s[0])
for s in self.getFilesToDelete():
os.remove(s)
#print "%s -> %s" % (self.baseFilename, dfn)
self.mode = 'a'
self.stream = self._open()
currentTime = int(time.time())
newRolloverAt = self.computeRollover(currentTime)
while newRolloverAt