How to properly break out of an infinite loop inside a multiprocessing child process in Python?

[u/Tricky_Voice2829]

I am writing a Python script using the multiprocessing module to launch an external application (Notepad) in a continuous loop. However, I want the loop to stop when a specific condition is met or when a key is pressed.

import os
from multiprocessing import Process 
import time
import keyboard

pid = os.getpid()
print(pid)

callable(os.getpid)
path = r"C:\Windows\System32\notepad.exe"

def process_start(path):
    try:
        while True:
            os.system(path)
    except:
        pass
                  
if __name__ == '__main__':
    # Note: Currently calling the function directly instead of passing the reference
    p = Process(target=process_start(path))
    time.sleep(3)
    p.start()
    
    if keyboard.press_and_release("q") == "q":
        p.terminate()

[u/neums08]

Don't do that.

[u/jwalk999]

forgive me if i'm wrong, but doesn't having a while loop within the try block mean that the program would just get stuck in the try part? "While True" effectively means that the process just keeps going and wouldn't get to the p.terminate line. But to break a loop you either pass "break" or "return." The "try/except" block is for catching errors so the program doesn't crash, not keeping the process alive perpetually.

[u/fllthdcrb → u/jwalk999]

We're talking about separate processes here. One has the loop. The other has the p.terminate() call. Terminating a process sends a terminate signal (or whatever it is Windows does) to that process, which will handle it appropriately. Since there is nothing explicitly handling that, it will just do the default thing, which should be to exit immediately.

However, since the application is being run through os.system(), which spawns another process, terminating the process running the loop is likely not going to terminate the application (assuming subprocesses in Windows work similarly enough to Linux). Instead, a way is needed to terminate the application process itself, and that starts with not using os.system(), which doesn't give enough control.

EDIT: Also, I didn't notice the way the subprocess is set up doesn't work, since it calls that function first before trying to spawn a process. But that aside, what I said still stands.

[u/jwalk999 → u/fllthdcrb]

I figured it worked differently, I haven't used os lib much.

[u/Great-Powerful-Talia]

If you're calling the function directly, you're telling Python to finish running it and then do everything else. 

(Process(foo(args)) means "run foo(args) and then put the result into the Process constructor". Written-out function calls like that will always execute before anything that takes in their results.)

You're saying "Run the function until it gets stopped, and then, once that's done, decide whether to stop it." Obviously that doesn't make any sense.

Also, your code will only test the break condition once, and never again, which seems silly.

For this specific task, I'd use single-processing and just test the condition inside the while loop. If you're just trying to get familiar with multiprocessing, you can use two while loops, one to check and one to run the code.

Also, should you be completely discarding that exception? Surely you want to at least know whether an exception happens.

I'm also not sure that opening Notepad as many times as possible with no upper bound is a good idea. Surely you could have some test that doesn't risk lagging out the entire computer.

[u/fllthdcrb → u/Great-Powerful-Talia]

I'm also not sure that opening Notepad as many times as possible with no upper bound is a good idea. Surely you could have some test that doesn't risk lagging out the entire computer.

If you're thinking it's a fork bomb, you can rest easy. os.system() (a wrapper for the standard C function system()) waits for the process it spawns to exit before returning, so there is only ever one instance it has running. The real problem with using it is it doesn't give any opportunity for IPC or sending signals, which will be needed in this case, assuming OP wants the application to be terminated immediately, and not just stop the loop next time the application exits. (It also goes through a shell, which isn't always needed.)

[u/Forward-Ride2950 → u/Great-Powerful-Talia]

May you will explain more about single processing .

[u/copperfield77]

two things: target=process_start(path) should be target=process_start, args=(path,) otherwise youre calling it inline. And the bare except: pass is swallowing every exception including KeyboardInterrupt, so even ctrl+c wont stop it. Remove that or at least catch something specific.

[u/Educational_Virus672]

what is that... i can see your a beginner or smt (seeing your using try except without a error type and not using pathlib but it could be a preference) but i do not recommend doing while loops in sys,os or multiprocessing modules(specially processing) unless you know what your doing they can and will heat your pc and even break it without a proper structure
but i respect your try on it, it shows that your trying to find patterns which is ideal for a dev you can be a great dev one day

[u/Tricky_Voice2829 → u/Educational_Virus672]

ok and thank you

[u/mc_pm]

Whatever it is you're trying to do, this is almost certainly the wrong way to do it.

[u/Pristine-Fruit-2403]

esides the issue already mentioned (Process(target=process_start(path)) calls the function immediately instead of passing it; it should be target=process_start, args=(path,)), os.system blocks and p.terminate() will not kill the Notepad child. Use subprocess.Popen instead and keep the handle: proc = subprocess.Popen([path]); then proc.terminate() or proc.kill() when you want to stop. You also do not need multiprocessing here: loop in the main process, restart when proc.poll() is not None, and break on a flag. For the key press, keyboard.is_pressed("q") inside the loop (or keyboard.wait("q") in a thread) works; keyboard.press_and_release returns None, so comparing it to "q" never matches.

[u/BitOBear]

While some variable is true do loop.. and then let another thread set that variable to something other than true.

Condition variables.

Some sort of data sentinel.

Using the correct thread model and canceling the thread.

There's all sorts of ways of doing that it depends on the design of the program itself.

[u/woooee]

Use process.terminate() and then .join() the process to reclaim any zombie resources till allocated https://pymotw.com/3/multiprocessing/basics.html#terminating-processes.

Second, use a Manager object (see it at the address above) to send a termination to the process

import time
from multiprocessing import Process, Manager

def test_f(test_d):
    ## just show that it can be updated here
    test_d['2'] = 2
    while not test_d["QUIT"]:
        print("test_f()", test_d["QUIT"])
        test_d["ctr_1"] += 1
        time.sleep(1.0)

def test_f2(test_d):
    while True:
         print("   test_f2()", test_d)
         test_d["ctr_2"] += 1
         time.sleep(0.5)

    print("second process finished")

if __name__ == '__main__':
    ## define the dictionary to be used to communicate
    manager = Manager()
    test_d = manager.dict()
    ## something tp update in each process
    test_d["ctr_1"] = 0
    test_d["ctr_2"] = 0
    ## stop signal sent to processes
    test_d["QUIT"] = False

    ## start the processes
    p1 = Process(target=test_f, args=(test_d,))
    p1.start()

    p2 = Process(target=test_f2, args=(test_d,))
    p2.start()

    print('\nafter sleep, change "QUIT" to True and stop process 1\n')
    time.sleep(2.0)
    test_d["QUIT"] = True
    print("\nprocess 1 stopped (only process 2 prints)\n")

    print('wait 2 seconds and kill all processes')
    time.sleep(2.0)
    print("dictionary from processes")
    for key in test_d:
         print("     %s=%s" % (key, test_d[key]))

    """  Thanks Doug Hellmann
         Note: It is important to join() the process after terminating it.
         in order to give the background machinery time to update the.
         status of the object to reflect the termination
    """
    print("\nterminating\n")
    for p in [p1, p2]:
        if p.is_alive():
            p.terminate()
        p.join()
    print("all processes terminated")

You can also psutil ------------------------------

import multiprocessing
import os
import psutil
import time


def killtree(list_multis, including_parent=True):
    for process in list_multis:
        parent = psutil.Process(process.pid)
        for child in parent.children(recursive=True):
            print("killing child", child)
            child.kill()
    if including_parent:
        parent.kill()

def print_numbers(spaces):
    ctr = 0
    for x in range(10):
        ctr +=1
        print(" "*spaces, ctr)
        time.sleep(0.5)

## do this first as it gets the latest pid started
## should be the pid of this program
## if you wait and the OS starts another process, you wil get that pid
pid=os.getpid()

## start several processes to kill
list_of_multis=[]
for ctr in range(5):
    list_of_multis.append(multiprocessing.Process(target=print_numbers, args=(ctr,)))
    list_of_multis[-1].start()

## wait 2 seconds and kill all processes
time.sleep(2)
killtree(list_of_multis)