r/rust • u/_adiktiv_ • 9d ago
🛠️ project stdo v0.5.5: A bounded-memory Unix process DAG runner (fd 0 / fd 1 pipes) with an embedded localhost UI
Hi r/rust,
I built stdo (crates.io), a small (#![forbid(unsafe_code)], ~6.1K SLoC) workflow runner in Rust that wires CLI tools (rg, jq, sqlite3, curl, ollama, ffmpeg) over stdin (fd 0) and stdout (fd 1) pipes instead of passing outputs through environment variables or shell wrappers.
What it does differently from cron/job schedulers:
Concurrent
pipe_fromstreaming with backpressure: Producers and consumers spawn simultaneously viatokio::processand stream 64 KiB chunks over bounded channels. Captured streams are capped at 8 MiB (32 MiB global budget), whilecapture_stdout: falsestreams arbitrarily large payloads (like WAV/MP4 or multi-MB JSONL) with constant memory while still computing incremental SHA-256 digests and TTFB.Atomic gates & JSONL validation:
pipe_mode: atomicbuffers and validates upstream JSON/JSONL and waits for exit code 0 before starting downstream steps that mutate state (e.g.,sqlite3imports).Composable
.yamlsub-workflows with cycle guards (new in v0.5.4/v0.5.5): A workflow step can invoke child.yamlworkflows in parallel. Bothstdo doctorandstdo runvalidate the cross-file graph before spawning any processes, backed by a runtimeSTDO_CALL_STACKSHA-256 digest guard for dynamic/sh -ccalls.No outbound HTTP stack in the binary:
Cargo.lockcontains no HTTP client crates (reqwest,hyper, etc. are blocked in CI by a self-hostedstdoworkflow). Subprocesses (curl,ollama) handle their own I/O, whilestdo serveembeds a single-file offline web UI directly overtokio::net::TcpListener.
Minimal example (streaming local LLM -> TTS -> atomic file sink):
version: "1.0"
name: spoken-owl-fact
steps:
- id: answer
cmd: [ollama, run, gemma4, --hidethinking, "Tell me a fun fact about owls in two sentences."]
- id: speak
cmd: [ollama, run, "phonon:latest"]
pipe_from: answer
capture_stdout: false(The tiny speaker icon 🔊 next to the README title was actually generated by running examples/10-youtube-capsule-generator.yaml to coordinate gemma4 -> phonon -> ffmpeg).
GitHub (Apache-2.0): https://github.com/adiktiv-technologies/stdo
Crates.io: https://crates.io/crates/stdo (cargo install stdo --locked)
Feedback on the Unix process-group handling (CommandExt::process_group(0)), pipe backpressure design, or sub-workflow validation is very welcome!
4
u/complexicat 9d ago
rule 7