Redis AOF刷新策略分析(转载)

binghai03

此文为转载，原文： http://afei2.sinaapp.com/?p=536



   redis支持使用aof来进行持久化，防止数据丢失，aof的刷新策略通过参数appendfsync控制，有三个值：always、everysec、no，默认是everysec。
   下面从源码的角度剖析一下aof的刷新策略。
   每次redis进入event循环准备执行这个event时，会调用beforeSleep方法


 
void aeMain(aeEventLoop *eventLoop) {
eventLoop->stop = 0;
while (!eventLoop->stop) {
if (eventLoop->beforesleep != NULL)
eventLoop->beforesleep(eventLoop);
aeProcessEvents(eventLoop, AE_ALL_EVENTS);
}
}
/* This function gets called every time Redis is entering the
* main loop of the event driven library, that is, before to sleep
* for ready file descriptors. */
void beforeSleep(struct aeEventLoop *eventLoop) {
......
/* Write the AOF buffer on disk */
flushAppendOnlyFile(0);
......
}
   上面的代码中的flushAppendOnlyFile(int force)进行实际的执行。


   
   src/aof.c
 

void flushAppendOnlyFile(int force) {
......
/* Perform the fsync if needed. */
if (server.aof_fsync == AOF_FSYNC_ALWAYS) {
/* aof_fsync is defined as fdatasync() for Linux in order to avoid
* flushing metadata. */
latencyStartMonitor(latency);
aof_fsync(server.aof_fd); /* Let's try to get this data on the disk */
latencyEndMonitor(latency);
latencyAddSampleIfNeeded("aof-fsync-always",latency);
server.aof_last_fsync = server.unixtime;
} else if ((server.aof_fsync == AOF_FSYNC_EVERYSEC &&
server.unixtime > server.aof_last_fsync)) {
if (!sync_in_progress) aof_background_fsync(server.aof_fd);
server.aof_last_fsync = server.unixtime;
}
}
   AOF_FSYNC_ALWAYS会调用aof_fsync进行同步写入，而aof_fsync在linux下就是fdatasync，


   AOF_FSYNC_EVERYSEC会调用aof_background_fsync，而aof_background_fsync会创建一个任务交给后台的bio线程进行处理。 

/* Define aof_fsync to fdatasync() in Linux and fsync() for all the rest */
#ifdef __linux__
#define aof_fsync fdatasync
#else
#define aof_fsync fsync
#endif
/* Starts a background task that performs fsync() against the specified
* file descriptor (the one of the AOF file) in another thread. */
void aof_background_fsync(int fd) {
bioCreateBackgroundJob(REDIS_BIO_AOF_FSYNC,(void*)(long)fd,NULL,NULL);
}
    其中everysec是通过下面的逻辑来进行的，检测后台是否fsync任务在进行，如果有的话，判断上次的fsync距离现在的时间，如果大于2s，则阻塞，否则直接进行后台队列。
    如果上一次的fsync执行了2s多，则会阻塞执行，直到写入成功，这个时候日志中会记录下面一条记录，并且增加info中对应的aof_delayed_fsync值
 
[5750] 12 Aug 09:56:17.057 * Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down Redis.
详细逻辑如下：


/*
* When the fsync policy is set to 'everysec' we may delay the flush if there
* is still an fsync() going on in the background thread, since for instance
* on Linux write(2) will be blocked by the background fsync anyway.
* When this happens we remember that there is some aof buffer to be
* flushed ASAP, and will try to do that in the serverCron() function.
*
* However if force is set to 1 we'll write regardless of the background
* fsync.
*
* 但是如果上一次的fsync执行了2s多，则会阻塞执行，直到写入成功
*/
/* With this append fsync policy we do background fsyncing.
* If the fsync is still in progress we can try to delay
* the write for a couple of seconds. */
if (sync_in_progress) {
if (server.aof_flush_postponed_start == 0) {
/* No previous write postponinig, remember that we are
* postponing the flush and return. */
server.aof_flush_postponed_start = server.unixtime;
return;
} else if (server.unixtime - server.aof_flush_postponed_start < 2) {
/* We were already waiting for fsync to finish, but for less
* than two seconds this is still ok. Postpone again. */
return;
}
/* Otherwise fall trough, and go write since we can't wait
* over two seconds. */


aof_pending_bio_fsync
/* Return the number of pending jobs of the specified type. */
bioPendingJobsOfType(REDIS_BIO_AOF_FSYNC),
serverCron中检查server.aof_flush_postponed_start，如果有的话，就追加一次flush，但是只有在上面的情况下会导致阻塞，其他情况下都会很快返回；
/* AOF postponed flush: Try at every cron cycle if the slow fsync
* completed. */
if (server.aof_flush_postponed_start) flushAppendOnlyFile(0);