Awhile back I had to use libevent for a high-performance application I was writing, and several times before that I had stumbled into it on various projects. I think its bloody awesome and its so awesome I decided to write a tutorial on it. So now your asking: what is libevent?

From the libevent website: The libevent API provides a mechanism to execute a callback function when a specific event occurs on a file descriptor or after a timeout has been reached. Furthermore, libevent also support callbacks due to signals or regular timeouts. libevent is meant to replace the event loop found in event driven network servers. An application just needs to call event_dispatch() and then add or remove events dynamically without having to change the event loop.

In other words, libevent can be used with systems that support select, epoll etc... to provide high-performance edge based event handling. One of the downsides is that it isn't ready yet for UDP sockets, but it does handle Unix Sockets, TCP, and provides facilities for DNS and HTTP. Libevent can handle numerous connections with minimal overhead and can provide the answer to the 10k problem.

If your familiar with C and network programming... game on!

Compiling and Installing libevent

Download, untar the source code and enter the directory:

wget github.com/downloads/libevent/libevent/libevent-2.0.20-stable.tar.gz
tar -xzvf libevent-2.0.20-stable.tar.gz
cd libevent-2.0.20-stable

Configure, make and install the libraries:

./configure
make
sudo make install

Libevent Echo Server Tutorial

One of the areas that I think Libevent is lacking in is useful beginner orientated tutorials that allow aspiring programmers to pick up this library. Its documentation is pretty good and well-defined, but I felt that it needed some clarification so hopefully this document can help other programmers out.

Libevent Basics

Libevent's most basic principle is that is runs off of events and these events are used by the event base. The event base uses these events through either/or listener and an event dispatcher. As a result, a good place for you - the programmer to start is knowing whether there will be read events and/or write events. Then you define your event callbacks and finally implement them. Its not really that complex - I promise.

Libevent Echo Unix Socket Server Example

One of the easiest ways to learn something is to create it and so you are going to create a unix socket echo server that implements Libevent. Open up your favorite code editor and build a skeleton C program - main and the usual suspect includes.

Add the following includes and define:

#include <event2/listener.h>
#include <event2/bufferevent.h>
#include <event2/buffer.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
#define UNIX_SOCK_PATH "/tmp/mysocket"

Now lets make your main look like this:

int main(int argc, char **argv)
{
    struct event_base *base;
    struct evconnlistener *listener;
    struct sockaddr_un sin;
 
    /* Create new event base */
    base = event_base_new();
    if (!base) {
	puts("Couldn't open event base");
	return 1;
    }
 
    /* Clear the sockaddr before using it, in case there are extra
     * platform-specific fields that can mess us up. */
    memset(&sin, 0, sizeof(sin));
    sin.sun_family = AF_LOCAL;
    strcpy(sin.sun_path, UNIX_SOCK_PATH);
 
    /* Create a new listener */
    listener = evconnlistener_new_bind(base, accept_conn_cb, NULL,
				       LEV_OPT_CLOSE_ON_FREE | LEV_OPT_REUSEABLE, -1,
				       (struct sockaddr *) &sin, sizeof(sin));
    if (!listener) {
	perror("Couldn't create listener");
	return 1;
    }
    evconnlistener_set_error_cb(listener, accept_error_cb);
 
    /* Lets rock */
    event_base_dispatch(base);
    return 0;
}

UHOH! main() is now referencing two functions that don't exist: accept_error_cb() and accept_conn_cb(). These two functions allow libevent to handle when a new connection is created and what to do when the listener encounters an error. Lets add these two functions - they are pretty explanatory.

static void
accept_conn_cb(struct evconnlistener *listener, evutil_socket_t fd, struct sockaddr *address, int socklen, void *ctx)
{
    /* We got a new connection! Set up a bufferevent for it. */
    struct event_base *base = evconnlistener_get_base(listener);
    struct bufferevent *bev = bufferevent_socket_new(base, fd, BEV_OPT_CLOSE_ON_FREE);
 
    bufferevent_setcb(bev, echo_read_cb, NULL, echo_event_cb, NULL);
 
    bufferevent_enable(bev, EV_READ | EV_WRITE);
}
 
static void accept_error_cb(struct evconnlistener *listener, void *ctx)
{
    struct event_base *base = evconnlistener_get_base(listener);
    int err = EVUTIL_SOCKET_ERROR();
    fprintf(stderr, "Got an error %d (%s) on the listener. "
	    "Shutting down.\n", err, evutil_socket_error_to_string(err));
 
    event_base_loopexit(base, NULL);
}

Again the accept_connection callback references two unknown functions as parameters in the bufferevent_setcb() function. Lets talk about this function a bit with an illustration from the Libevent documentation:

void bufferevent_getcb(struct bufferevent *bufev,
    bufferevent_data_cb *readcb_ptr,
    bufferevent_data_cb *writecb_ptr,
    bufferevent_event_cb *eventcb_ptr,
    void **cbarg_ptr);

As you can see, it requires a bufferevent, a callback for read events, a callback for write events, a callback for an event and a parameter for callback args. I've mentioned callbacks a few times so I'm sure you either know what it is, or have googled it by now ;) From the documentation a Bufferevent is:

 A "bufferevent" consists of an underlying transport (like a socket), a read buffer, and a write buffer. Instead of regular events, which give callbacks when the underlying transport is ready to be read or written, a bufferevent invokes its user-supplied callbacks when it has read or written enough data.
Every bufferevent has an input buffer and an output buffer. These are of type "struct evbuffer". When you have data to write on a bufferevent, you add it to the output buffer; when a bufferevent has data for you to read, you drain it from the input buffer.

Now you know what buffer events are and their general usage. Add the following two functions:

static void echo_read_cb(struct bufferevent *bev, void *ctx)
{
    /* This callback is invoked when there is data to read on bev. */
    struct evbuffer *input = bufferevent_get_input(bev);
    struct evbuffer *output = bufferevent_get_output(bev);
 
    size_t len = evbuffer_get_length(input);
    char *data;
    data = malloc(len);
    evbuffer_copyout(input, data, len);
 
    printf("we got some data: %s\n", data);
 
    /* Copy all the data from the input buffer to the output buffer. */
    evbuffer_add_buffer(output, input);
    free(data);
}
 
static void echo_event_cb(struct bufferevent *bev, short events, void *ctx)
{
    if (events & BEV_EVENT_ERROR)
	perror("Error from bufferevent");
    if (events & (BEV_EVENT_EOF | BEV_EVENT_ERROR)) {
	bufferevent_free(bev);
    }
}

The above functions should be fairly self explanatory, but a possible stumbling block for new programmers is how to get data out of the Libevent read buffer in this example - did you see this: evbuffer_copyout(input, data, len); I won't explain it in detail, but the method calls can be found at http://www.wangafu.net/~nickm/libevent-book/Ref7_evbuffer.html

Essentially you should know that evbuffer_copyout copies data AND DOESN'T remove the data. The function evbuffer_remove_buffer() copies AND removes data from the buffer.

Now that you have all of the pieces of the program lets compile it and test it

gcc -Wall -levent yourprogram.c -o server
./server

Connect to your program using netcat in a second terminal- you should see output echo'd back to you & data on the serverside ;)

echo -n 'test' | nc -U /tmp/mysocket1

Further reading

• http://www.libevent.org
• http://www.wangafu.net/~nickm/libevent-book/Ref7_evbuffer.html