Recently, I was using Apache Thrift<\/a> for a distributed high performance C++ application. Apache Thrift is an RPC<\/a> for calling functions on other ends of networks and across different languages. On a single thread, using an RPC is a bad idea, if performance is the goal. So the way to get high performance from such a system is by putting multiple clients in many threads. However, the problem is that Apache Thrift’s clients are not thread-safe! What do we do?<\/p>\n\n\n\n The solution is an object pool of clients, and since the application is a high-performance, I chose it to be a lock-free object pool. Lock-free means that no mutexes are involved in the design, but only atomic variables. I couldn’t find anything out there, so I implemented my own, which I hope will be useful to others, too. While this design is lock-free, it’s not “wait-free”. Meaning that, obviously, if all objects in the pool are used, the system will block\/wait until an object is available, or the programmer has to setup an asynchronous waiting mechanism. It’s possible to “try” to get an object in this design.<\/p>\n\n\n\n In the following I’ll discuss my design of this object pool.<\/p>\n\n\n\n The design of this object pool is simple. <\/p>\n\n\n\n Shall we look at some code?<\/p>\n\n\n\n As you can see, the class is not copyable or movable, for simplicity. Besdies that, the following methods can be seen:<\/p>\n\n\n\n You can find the full implementation here.<\/a><\/p>\n\n\n\n Here’s a simple design of a lock-free object pool. No more than 200 lines. The feature I find cool in this object pool is that the objects themselves are not being passed around, but only their indices, which can be done quite efficiently on most modern CPUs.<\/p>\n","protected":false},"excerpt":{"rendered":" Introduction Recently, I was using Apache Thrift for a distributed high performance C++ application. Apache Thrift is an RPC for calling functions on other ends of networks and across different languages. On a single thread, using an RPC is a bad idea, if performance is the goal. So the way to get high performance from … Continue reading A simple, lock-free object-pool<\/span> The design<\/h2>\n\n\n\n
BorrowedObject<\/code>” will be returned, which contains a pointer to the borrows object. This is a safe mechanism, where returning to the pool is done automatically when this object is destroyed.<\/li><\/ul>\n\n\n\ntemplate <typename T>\nclass ObjectPool\n{\n AvailableObjectsQueueType availableObjectsQueue;\n std::size_t objectCount;\n std::vector<T> objects;\n std::atomic_uint64_t availableObjectsCount;\n std::atomic_bool shutdownFlag;\n\npublic:\n friend BorrowedObject<T>;\n ObjectPool(std::size_t Size, std::function<T(std::size_t)> objectInitializers);\n ObjectPool() = delete;\n ObjectPool(const ObjectPool&) = delete;\n ObjectPool(ObjectPool&&) = delete;\n ObjectPool& operator=(const ObjectPool&) = delete;\n ObjectPool& operator=(ObjectPool&&) = delete;\n\n BorrowedObject<T> borrowObj(uint64_t sleepTime_ms = 0);\n BorrowedObject<T> try_borrowObj(bool& success);\n std::size_t size() const;\n uint64_t getAvailableObjectsCount() const;\n std::vector<T>& getInternalObjects_unsafe();\n void shutdown();\n ~ObjectPool();\n};<\/code><\/pre>\n\n\n\nstd::size_t<\/code>. The purpose of this is to be able to have the initialization done as a function of the number of the object in the pool.<\/li>borrowObj()<\/code>: A blocking method for borrowing an object from the pool.<\/li>try_borrowObj()<\/code>: A non-blocking method for borrowing an object from a the pool, returns a null object on failure.<\/li>shutdown()<\/code>: Shutdown the object pool, which calls the destructor of all objects and prevents borrowing further.<\/li><\/ul>\n\n\n\nConclusion<\/h2>\n\n\n\n