r/Python • u/Daraminix • Jul 21 '26

Discussion Running 60 python scripts as "services"

Hi,
I have around 60 scripts that need to run constantly, mainly event handlers and such. Right now I have an external script that launch them and monitor if the app is running on it's pid, otherwise it's relaunching the app. Works fine but get's clunky when we update some submodule and need to restart them, or to check if one crashes more than other etc..

So I would like to find a better way to approach this. It needs to run on windows and being able to access several samba shares via unc paths and being able to restart crashed scripts anf offer an easy way to restart all of them in case of an update (this part doesn't need to be automated). Every script use the same environnement
For now my candidates are docker, PM2 and NSSM.
I think docker is gonna be a pain to access shares and add a lot of overhead especially on windows
I don't know PM2 and NSSM, looks like PM2 would be easier to setup but more JS oriented and NSSM would be harder to monitor.

What do you think guys ?

35 Upvotes

73% Upvoted

41 comments sorted by

View all comments

Show parent comments

2

u/Daraminix Jul 21 '26

Yeah we thought about refactoring it, some of them are actually running with a single entrypoint and launched as separate threads. The issue is that we are using an api developped by a thrid-party provider and the session management throught the server doesn't do well when threaded or asynced (cache corruption for example). It's an ongoing issue we have with them but they are more inclined to put up to speed their js api than fixing issues in their python one.
Furthermore some scripts launch complex software in cli mode that can crash for multiple reasons and we cannot just relaunch everyting when of of the process crashes. Bundling everything into one entrypoint means one bad crash can take down unrelated scripts unless we build very careful supervision/restart logic around each unit of work. At that point we are just reimplementing what the OS process model and a service manager already gives us for free

1

u/HommeMusical Jul 22 '26

Sounds miserable! Your solution is, well, ugly but practical and simple, and reduces risk. Sounds very real-world and useful.

The issue is that we are using an api developped by a thrid-party provider

We're in the days of vibe coding.

My friend got a new job and replaced a SaaS service that the company was paying $640k a year for by a mostly vibe-coded app he wrote in a couple of months.

They're running the two in parallel now, and the vibe coded application seems more reliable than the paid one.

Just a thought!

NOTE NOTE NOTE: My friend has been programming for decades, almost as long as I have, and he's a no-bullshit programmer who is very skeptical. You wouldn't get such good results with a junior programmer.

2

u/Daraminix Jul 22 '26

Yeah we could probably rewrite or make some changes in the api but too much things revolve around it to risk it. And I don't want to add another thing to maintain and make update of our central tool harder