Is there a way to stop a running process in concurrent.futures?

Question

How can I terminate a running process, started using concurrent.futures? As I understand, the cancel() method is there to remove a process from the queue if it is not running. But what about killing a running process? For example, if I have a long running process, and I want to stop it when I press a Cancel button in a GUI.

Answer 1

In this case it would probably be better to use a multiprocessing.Process for a long running task.

Create a multiprocessing.Event before starting the new process. Have the child process check the status of this Event regularly, and make it exit when Event.is_set() returns True.

In your GUI code, have the callback associated with the button call set() on the Event.

Answer 2

You may want to look at my answer to a related StackOverflow question here.

In short, there does not appear to be a simple way to cancel a running process inside a concurrent.futures.ProcessPoolExecutor. But you can accomplish it in a hacky way by killing the child processes manually.

Answer 3

You can use the _processes from the executor.

For example script.py:

import signal
import time
from concurrent.futures import ProcessPoolExecutor


def sleep_squre(x):
    def sigterm_handler(signum, frame):
        raise SystemExit(signum)

    signal.signal(signal.SIGTERM, sigterm_handler)
    try:
        time.sleep(x)
    except SystemExit:
        return -1
    return x * x

with ProcessPoolExecutor(max_workers=2) as ex:
    results = ex.map(sleep_squre, [1, 5])
    time.sleep(3)
    for pid, proc in ex._processes.items():
        proc.terminate()
print(list(results))

In this case, we send SIGKILL after 3 seconds to all processes.

$ python3 script.py
[1, -1]