reuse thread pool after calling stop(true) ?

[shikharkhattar]

Hi,

If I try to reuse the thread pool (using push) after calling stop(true), it doesn't create any new threads ? Something like the following sequence:

auto thread_pool  = new ctpl::thread_pool(2);
for (int i = 0; i < 10; i++)
{
    thread_pool->push(func, i);
}  
thread_pool->stop(true);
thread_pool->resize(4);
/* Wait a while */
thread_pool->push(func, i); // ==> func is not executed

Is this behavior intentional ? I have gotten around this by destroying the old pool and creating a new pool but that does not seem efficient.

[wangxiao1254]

I have the same question. Or to rephrase it:
Is there any way to call "join" to all functions in the pool and queue without deleting the threads?

[zyvitski]

From reading over the source, the simplest fix would be to add a start() function like this

void start(int nThreads){
    this->init();
    this->resize(nThreads);
}

This effectively emulates the behavior of the constructor:

    `thread_pool(int nThreads, int queueSize = _ctplThreadPoolLength_) : q(queueSize) { this->init(); this->resize(nThreads); }`

Which would allow you to restart the pool by effectively rebuilding the threads.

While I feel this would work, it would be a cheap fix to the underlying problem.

In a more ideal implementation the resize() function would be written in a manor that would allow the resize to be performed without stopping all work. This would involve synchronizing the changes to the thread container with the workers themselves, allowing the workers to pause working instead of going to a full stop while threads are being added or removed.

[wangxiao1254]

Does it delete and create all threads again?
Ideally we do not want unnecessary system calls to create threads.

[zyvitski]

The bit of code I provided is the naive approach to solving the problem. It would wind up having to delete and rebuild the threads. See the last paragraph of my original comment.

A better solution would be a system that that can pause work on all of the threads and only kill the ones you that are needed. The issue isn't with the functionality of stop(). The issue is what it is being used for. It is being used before calling resize() to allow safe resizing of the pool.

It would be better to have another function that could pause work or have resize be written in a way that it can resize without having to stop work at all.