Redis VS Oracle Advance Queue性能对比 (二)

周翔

　　本文Redis VS Oracle Advance Queue性能对比 (一)  的续篇。本文主要关注于多线程下面的比较。在真实的应用环境下面多个程序多个线程同时出入队的情况比较普篇，所以我们更需要看看在这样的情况下的性能对比。Oracle Advance Queue的底下是数据库的表，我们知道对同一个表太多的Session进行操作会有些锁的问题，所以在我们的测试中既有单个Queue的多线程对比，也有多个线程对多个Queue同时操作的对比。
　　对于Redis本身是个单进程单线程的服务程序。根据作者的建议，如果要发挥多个core (CPU)服务器的功效，我们需要开启多个实例。本文的测试服务器是一个八核的Linux,所以我们选择开启8个实例来服务8个独立的队列来对比Oracle Advance Queue的多表多队列。
　　这里是Redis作者原文的引用。
　　


Redis is single threaded, how can I exploit multiple CPU / cores?
　　Simply start multiple instances of Redis in different ports in the same box and threat them as different servers! Given that Redis is a distributed database anyway in order to scale you need to think in terms of multiple computational units. At some point a single box may not be enough anyway.
　　In general key-value databases are very scalable because of the property that different keys can stay on different servers independently.
　　In Redis there are client libraries such Redis-rb (the Ruby client) that are able to handle multiple servers automatically using consistent hashing. We are going to implement consistent hashing in all the other major client libraries. If you use a different language you can implement it yourself otherwise just hash the key before to SET / GET it from a given server. For example imagine to have N Redis servers, server-0, server-1, ..., server-N. You want to store the key "foo", what's the right server where to put "foo" in order to distribute keys evenly among different servers? Just perform the crc = CRC32("foo"), then servernum = crc % N (the rest of the division for N). This will give a number between 0 and N-1 for every key. Connect to this server and store the key. The same for gets.
　　This is a basic way of performing key partitioning, consistent hashing is much better and this is why after Redis 1.0 will be released we'll try to implement this in every widely used client library starting from Python and PHP (Ruby already implements this support).
　　
　　测试代码的设计。
　　
　　



static void MultiThreadEnqueue(List queues, List messages, string queueType)
{
Dictionary queueTaskDic = new Dictionary();
var sw = Stopwatch.StartNew();
int i = 1;
foreach (var q in queues)
{
queueTaskDic[q] = new Task(Enqueue, new QueueTask { Queue = q, Messages = messages });
queueTaskDic[q].Start();
i++;
}

Task.WaitAll(queueTaskDic.Values.ToArray());
sw.Stop();
Console.WriteLine("{2}, Enqueue {0} messages cost {1} ms, {4}, {0}, {3:0.0000}", messages.Count() * queues.Count(), sw.ElapsedMilliseconds, queueType, messages.Count() * queues.Count() * 1000.0 / sw.ElapsedMilliseconds, queues.Count());
}
static void MultiThreadDequeue(List queues, int msgCount, string queueType)
{
Dictionary queueTaskDic = new Dictionary();
var sw = Stopwatch.StartNew();
foreach (var q in queues)
{
queueTaskDic[q] = new Task(Dequeue, new QueueTask { Queue = q, DequeueCount=msgCount });
queueTaskDic[q].Start();
}

Task.WaitAll(queueTaskDic.Values.ToArray());
sw.Stop();
Console.WriteLine("{2}, Dequeue {0} messages cost {1} ms, {4}, {0}, {3:0.0000}", msgCount * queues.Count(), sw.ElapsedMilliseconds, queueType, msgCount * queues.Count() * 1000.0 / sw.ElapsedMilliseconds, queues.Count());
}

　　根据传入的Queue数组的个数来建立多个Task(线程)，在些线程中做入队或出队的操作。然后等待所有的线程完成，这个计数的时间就是取所有线程中花的时间最长的一个。
　　
　　



AQAccessor.cs
public static List CreateSharedTablePersistentQueues(int queueCount)
{
List ret = new List();
for (int i = 0; i < queueCount; i++)
{
ret.Add(CreatePersistentQueue(1));
}
return ret;
}

public static List CreateSharedTableBufferedQueues(int queueCount)
{
List ret = new List();
for (int i = 0; i < queueCount; i++)
{
ret.Add(CreateBufferedQueue(1));
}
return ret;
}
public static List CreateDedicatedTableBufferedQueues(int queueCount)
{
List ret = new List();
for (int i = 0; i < queueCount; i++)
{
ret.Add(CreateBufferedQueue(i % 8 +1));
}
return ret;
}
public static List CreateDedicatedTablePersistentQueues(int queueCount)
{
List ret = new List();
for (int i = 0; i < queueCount; i++)
{
ret.Add(CreatePersistentQueue(i % 8 + 1));
}
return ret;
}

RedisAccessor.cs


public static List CreateSharedServerQueues(int queueCount)
{
List ret = new List();
for (int i = 0; i < queueCount; i++)
{
ret.Add(CreateSharedServerQueue(1));
}
return ret;
}
public static List CreateDedicatedServerQueues(int queueCount)
{
List ret = new List();
for (int i = 0; i < queueCount; i++)
{
ret.Add(CreateDedicatedServerQueue(i % 8 + 1));
}
return ret;
}





　　共享的队列的意思是所有的操作多将在queue1 （对于Oracle Advance Queue来说，对应的表是queue_table1， 对于Redis来说对应于6380端口上的服务）上完成，CreateSharedTablePersistentQueues(), CreateSharedTableBufferedQueues()分别构建的是一群共享一个表的存储或内存Queue。而CreateSharedServerQueues()构建的是一群公用一个端口的Queue。
　　

　　独立的队列的意思是操作是分布在8个不同的Oracle表或8个不同端口的Redis服务器上的。如果需要构建的线程超过8，按照取8的模来分布到这八个队列上。CreateDedicatedTableBufferedQueues()和CreateDedicatedTablePersistentQueues（）
　　是为Oracle AQ构建一群队列的。CreateDedicatedServerQueues()是为Redis 构建队列的。
　　测试代码和上篇是同一个。
　　


测试结果：



　　测试结果原始数据可以从http://files.iyunv.com/ivenxu/multthreadQueueResult.zip下载。
　　

• 测试结果中，Oracle永久存储队列（Oracle Persistent AQ）无论在共享表还是在独立表的情况下，性能没有随着线程的增加而提高。在每秒完成20到30之间。相对于其他类型来说，差别太大（在一两个数量级）。
  
• 共享队列下的对比(Oracle Shared Buffered AQ vs Redis Shared Server Queue)
  

　　





　　


　　　　　　　　　　　　这里的线程数目是指入队和出队个几个。比如2的意思是有2个入队线程又有2个出队线程

• 　　Oracle 缓存队列和Redis基本上在同一数量级
  
• 　　Redis比Oracle缓存队列无论在如队和出队上都有更好的性能
  
• 　　Redis和Oracle缓存队列都在2到8个线程的时候达到了最佳的性能
  

• 独立队列下的对比(Oracle Dedicated Buffered AQ vs Redis Dedicated Server Queue)
  

　　





　　





　　　　Redis独立队列的情况下，在4个线程的情况下大到了最大的性能输出，之后却明显下降。

• Redis下面共享对立与独立队列的对比(Redis Shared Server Queue vs Redis Dedicated Server Queue)