Introduction

Recently, I was using Apache Thrift for a distributed high performance C++ application. Apache Thrift is an RPC for calling functions on other ends of networks and across different languages. On a single thread, using an RPC is a bad idea, if performance is the goal. So the way to get high performance from such a system is by putting multiple clients in many threads. However, the problem is that Apache Thrift’s clients are not thread-safe! What do we do?

The solution is an object pool of clients, and since the application is a high-performance, I chose it to be a lock-free object pool. Lock-free means that no mutexes are involved in the design, but only atomic variables. I couldn’t find anything out there, so I implemented my own, which I hope will be useful to others, too. While this design is lock-free, it’s not “wait-free”. Meaning that, obviously, if all objects in the pool are used, the system will block/wait until an object is available, or the programmer has to setup an asynchronous waiting mechanism. It’s possible to “try” to get an object in this design.

In the following I’ll discuss my design of this object pool.

The design

The design of this object pool is simple.

• There’s a vector of size N that holds N initialized objects. The initialization of the objects is done in the constructor, by passing a functor/lambda that intializes every object and puts them in the vector (hence, the objects must be at least move-constructible).
• I used a lock-free queue from boost. The queue has integers ranging from 0 to N-1. In order to check whether an object is available, its index in the vector has to be in this queue.
• If borrowing is successful, a new object “BorrowedObject” will be returned, which contains a pointer to the borrows object. This is a safe mechanism, where returning to the pool is done automatically when this object is destroyed.

Shall we look at some code?

template <typename T>
class ObjectPool
{
    AvailableObjectsQueueType availableObjectsQueue;
    std::size_t               objectCount;
    std::vector<T>            objects;
    std::atomic_uint64_t      availableObjectsCount;
    std::atomic_bool          shutdownFlag;

public:
    friend BorrowedObject<T>;
    ObjectPool(std::size_t Size, std::function<T(std::size_t)> objectInitializers);
    ObjectPool()                  = delete;
    ObjectPool(const ObjectPool&) = delete;
    ObjectPool(ObjectPool&&)      = delete;
    ObjectPool& operator=(const ObjectPool&) = delete;
    ObjectPool& operator=(ObjectPool&&) = delete;

    BorrowedObject<T> borrowObj(uint64_t sleepTime_ms = 0);
    BorrowedObject<T> try_borrowObj(bool& success);
    std::size_t       size() const;
    uint64_t    getAvailableObjectsCount() const;
    std::vector<T>&   getInternalObjects_unsafe();
    void              shutdown();
    ~ObjectPool();
};

As you can see, the class is not copyable or movable, for simplicity. Besdies that, the following methods can be seen:

• The constructor: Takes the size of the pool, and a functor/lambda to initialize the objects. The parameter in this functor/lambda is of type std::size_t. The purpose of this is to be able to have the initialization done as a function of the number of the object in the pool.
• borrowObj(): A blocking method for borrowing an object from the pool.
• try_borrowObj(): A non-blocking method for borrowing an object from a the pool, returns a null object on failure.
• shutdown(): Shutdown the object pool, which calls the destructor of all objects and prevents borrowing further.

You can find the full implementation here.

Conclusion

Here’s a simple design of a lock-free object pool. No more than 200 lines. The feature I find cool in this object pool is that the objects themselves are not being passed around, but only their indices, which can be done quite efficiently on most modern CPUs.