Fast polling using C, memcached, nginx and libevent

昊漫玉

　　很不错的方案，值得参考，如果能在nginx源码级别，尽兴模块定制，我想可能效果更好：）
　　引用地址http://amix.dk/blog/post/19414#Fast-polling-using-C-memcached-nginx-and-libevent
　　In this post I'll show you how to implement really fast polling using C and libevent, memcached and nginx. The performance of the server is over 2400 request pr. second on a not optimized Mac Book - that's 144.000 requests pr. minute.
　　At Plurk we use polling and we have thousands of live users hammering the service with poll requests. It's beginning to be pretty expensive so I set a goal to optimize it. We currently use this approach in production and it uses around 2% of CPU and _very_ little memory.
Choosing the stack
　　I could have chosen different stacks, but I chose following components:

• C and libevent: C is as low as you get (if you don't want to code in assembler) and libevent implements asynchronous IO for C. libevent also features a pretty nifty HTTP library for building scaleable HTTP servers. libevent is used in memcached.
  
• memached: Proven by tons of startups like Facebook. memcached is lightweight, scaleable and extremely optimized.
  
• nginx: Russian engineering when it's best. Enough said.
  

　　All these technologies are non-blocking and have proven their performance and scalability.
The architecture of fast polling
　　This is the architecture of the fast polling:

　　The thing to note is that we will only hit the Python cores if the cache is empty. This means that we populate the cache with an empty value when we don't have any new stuff for the client.
Hardest part, configuration of nginx
　　The polling has to support POST and GET requests and it seems that nginx favors GET requests. A problem I encountered was following:

• nginx's error_page handler strips the POST data that is sent with the POST request...
  

　　This was a major problem and it took a bit hacking and reading of nginx's mailing list to figure out a solution. After some time and experimenting I found it:
　　

location /Poll/ {
    proxy_pass http://localhost:8080/;
    error_page 501 404 502 = @fallback;
}
location @fallback {
    proxy_pass http://localhost:14002;
}　　
　　This means that we will first proxy to http://localhost:8080/, if this fails, we will use @fallback. In greater detail:

• on cache hit, we simply proxy redirect to the poll server and return the result
  
• on cache miss, we proxy to the Python core
  

　　Problem solved!
　　I didn't use nginx'es memcached module since it only supports one memcached server and only GET requests.
Benchmark
　　Before I show you the C code, I want to show you an Apache ab benchmark (around 2400 request pr. second on a Mac Book):
　　

amixs-macbook:~ amix$ ab -c 50 -n 5000 http://127.0.0.1/Poll/getMessages
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests

Server Software:        nginx/0.7.44
Server Hostname:        127.0.0.1
Server Port:            80
Document Path:          /Poll/getMessages
Document Length:        5 bytes
Concurrency Level:      50
Time taken for tests:   2.068 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Total transferred:      735000 bytes
HTML transferred:       25000 bytes
Requests per second:    2417.43 [#/sec] (mean)
Time per request:       20.683 [ms] (mean)
Time per request:       0.414 [ms] (mean, across all concurrent requests)
Transfer rate:          347.03 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.6      0       7
Processing:     5   20   3.2     20      39
Waiting:        4   20   3.1     19      39
Total:          6   21   3.0     20      39
Percentage of the requests served within a certain time (ms)
  50%     20
  66%     21
  75%     21
  80%     22
  90%     25
  95%     26
  98%     29
  99%     32
100%     39 (longest request)　　
　　Why not use Comet?
　　Comet is basically server pushing out updates. Comet may be the new buzz word, but I still think polling is easiest to scale and implement. From what I have seen, all comet solutions are very hacky, quirky and very hard to scale. The scaling is especially hard since the whole web-platform is built around the request-response model. This may change in the future, but until then I think polling is the safest way to go right now.
The C code
　　This is some of my first C code, so please report if you spot any problems or have suggestions. I actually enjoyed hacking this C program together!
　　#include <stdio.h>

#include <sys/types.h>
#include <sys/time.h>
#include <sys/queue.h>
#include <string.h>
#include <stdlib.h>
#include <err.h>
#include <event.h>
#include <evhttp.h>
#include <libmemcached/memcached.h>

/*
* Global
*/
typedef struct {
    char* host;
    int port;
} MServer;
int NUM_OF_SERVERS = 0;
MServer memcache_servers[4];
memcached_st *tcp_client;

/*
* Memcached info
*/
void add_mserver(char* host, int port) {
    memcache_servers[NUM_OF_SERVERS].host = host;
    memcache_servers[NUM_OF_SERVERS].port = port;
    NUM_OF_SERVERS++;
}
void init_memcache_servers() {
    add_mserver("localhost", 11211);
    add_mserver("192.168.0.51", 11211);
    //Add to memcached client
    tcp_client = memcached_create(NULL);
    int i;
    for(i=0; i < NUM_OF_SERVERS; i++) {
        memcached_server_add(tcp_client,
                             memcache_servers.host,
                             memcache_servers.port);
    }
}

/*
* Takes an `uri` and strips the query arguments.
*/
char* parse_path(const char* uri) {
    char c, *ret;
    int i, j, in_query = 0;
    ret = malloc(strlen(uri) + 1);
    for (i = j = 0; uri != '\0'; i++) {
        c = uri;
        if (c == '?') {
            break;
        }
        else {
            ret[j++] = c;
        }
    }
    ret[j] = '\0';
    return ret;
}

/*
* Checks if the path is in memcached, if not
* the connection gets dropped without an answer
* this is done so a proxy redirection
* can be dropped in nginx.
*/
void memcache_handler(struct evhttp_request *req, void *arg) {
    struct evbuffer *buf;
    buf = evbuffer_new();
    if (buf == NULL) {
        err(1, "failed to create response buffer");
    }
    char key[200];
   
    strcpy(key, "/Poll");
    char *request_uri = parse_path(req->uri);
    //Ensure no buffer overflows
    if(strlen(request_uri) > 125) {
        evhttp_connection_free(req->evcon);
    }
    else {
        strcat(key, request_uri);
        /* Fetch from memcached */
        memcached_return rc;
        char *cached;
        size_t string_length;
        uint32_t flags;
        cached = memcached_get(tcp_client, key, strlen(key),
                               &string_length, &flags, &rc);
        /* Return a result to the client */
        if(cached) {
            evbuffer_add_printf(buf, "%s", cached);
            evhttp_send_reply(req, HTTP_OK, "OK", buf);
            free(cached);
        }
        else {
            evhttp_connection_free(req->evcon);
        }
    }
    free(request_uri);
    evbuffer_free(buf);
}

int main(int argc, char **argv) {
    init_memcache_servers();
    struct evhttp *httpd;
    event_init();
    httpd = evhttp_start(argv[1], atoi(argv[2]));
    evhttp_set_gencb(httpd, memcache_handler, NULL);
    event_dispatch();
    evhttp_free(httpd);
    memcached_free(tcp_client);
    return 0;
}　　I was a bit lazy to create a config reader, mostly because in Python I would have written open("server.config").readlines(), in C it seems a bit more cumbersome :-p
　　As always, happy hacking and I hope you enjoyed this post!