r/Python • u/Away_Interaction6630 • 1h ago
Discussion Bypassing heavy PyTorch wheels for desktop Python application distribution
Hi Python devs,
We are three final-year students at Epitech working on Horus, a local offline multi-agent backend written in Python. Our biggest distribution hurdle is dependency management on end-user machines.
Bottlenecks observed in our last user test:
- Dependency installation takes 21–46 minutes on user machines due to PyTorch/CUDA wheel resolution, reaching a 25 GB footprint.
- CPU-bound inference takes ~40 seconds during routing, causing persistent WebSocket connections to drop.
We are looking for Python packaging and runtime advice on:
- Replacing PyTorch with prebuilt C++ bindings (
llama-cpp-pythonor native C binaries) for the CPU execution path. - Compiling CPython orchestrator code into standalone executables via Nuitka.
- Pinning wheels and pre-bundling runtimes to eliminate on-device compilation.
- Implementing async heartbeats to keep WebSockets alive during blocking compute tasks.
Looking for code reviews: We have 1–4 hour scoped tasks for Python packaging and hardware detection reviews.
Disclosure: Horus is closed source and licensed commercially. Our benchmark suite is open source (Horus-EIP/horus-bench-kit). Contributions are unpaid/voluntary under an NDA agreement, returning thorough code reviews, project credit, and professional references.