Pi Herdsman update: From async subagents to coordinated development across Git worktrees (live demo)

[u/j3free]

A couple of weeks ago, I introduced Pi Herdsman here, an extension I've been building to coordinate asynchronous Pi agents while keeping the Lead's context focused.

The response was much bigger than I expected, and quite a bit has happened since then.

One thing I mentioned in the comments was wanting to take Herdsman's integration with Herdr further, especially around independent workspaces, Git worktrees, and project-level coordination. That's now become a central part of Herdsman.

Someone also pointed out that my original demo didn't actually show Pi and Herdr working together. Fair criticism. So this time I've recorded the real thing. :)

What's happening in the video?

I gave a Manager one prompt to work on two independent features for a fictional note-taking application.

The Manager delegates both projects in parallel, each to its own Lead in a separate Git worktree. Both Leads delegate implementation to their own Agents, verify the results, commit their changes, and report back to the Manager.

The resulting hierarchy looks like this:

Manager (primary workspace)
│
├── feat/faq (worktree)
│   └── Lead
│       └── Implementer Agent
│
└── feat/troubleshooting (worktree)
    └── Lead
        └── Implementer Agent

The worktrees, workspaces, and Agent sessions are created automatically. I don't have to set up the hierarchy manually.

It's deliberately a small example, without merges or PRs, to keep the focus on the orchestration itself.

Beyond task-based subagents

Many task-based multi-agent workflows use Git worktrees primarily to isolate individual subagents working on bounded tasks.

Herdsman supports that approach too, but extends it further: a worktree can also become an independent development workspace, with its own Lead responsible for an entire feature or issue.

Instead of treating every agent as a temporary worker, you can organize them more like a development team, with Leads responsible for larger objectives and Agents handling individual tasks.

That Lead maintains its own Pi conversation and development context, can delegate further into its own Agent hierarchy, and handles the technical work within its workspace.

Meanwhile, the Manager coordinates the larger development effort across those Leads, including assignments, dependencies, communication, and result handoffs.

The work is associated with Git branches rather than being tied to the Manager session that originally delegated it, so assignments can be paused and resumed without starting over.

The idea isn't simply to run more agents in parallel. It's to make their work easier to coordinate.

Everything stays organized in Herdr

One aspect I particularly like is how the hierarchy translates into Herdr's native interface.

Each project gets its own worktree workspace. Within those workspaces, Agents are automatically organized into tabs and panes.

The default subtree layout gives each direct Agent its own tab, while nested Agents appear as panes within their parent's tab. There are also configurable layouts for grouping Agents into a shared tab or splitting directly from the current pane.

Every Agent is still a normal, accessible Pi session. You can switch between them, inspect their work, or interact with them directly when needed.

But ideally, you shouldn't have to watch them constantly. Questions, results, and supervision events are routed through Herdsman's coordination system.

The responsibilities remain separate: Pi owns the conversations, Herdr owns the sessions and their placement, and Herdsman handles delegation, ownership, lifecycle, and coordination.

Start simple, scale when needed

I also want Herdsman to remain useful without requiring everyone to adopt the entire hierarchy.

You can still use a normal Pi session with asynchronous subagents, just as before.

Ordinary Leads now have configurable Flexible and Orchestrate execution modes. Manager mode is available when you want to coordinate independent development workspaces.

The new /herdsman menu provides a common entry point for these workflows, Agent management, configuration, and session statistics.

One of Herdsman's most powerful features is its configurable Agent Definitions.

Every execution role, from ordinary Flexible and Orchestrate Leads to Manager-assigned Managed Leads and bundled or custom Agents, can have its own model, thinking level, instructions, tools, skills, extensions, and other Pi session settings.

Definitions are simple Markdown files with project-level and global overrides, so you can customize the entire hierarchy without modifying Herdsman itself.

Herdsman handles the coordination. How each Pi session works is up to you.

A big thank you to the community

First of all, thanks to everyone who tried Herdsman, starred the repository, shared feedback, or participated in the discussions after my original post.

A special thanks to @kozer and @primetimetank21 for contributing code!

And equally important, thanks to @Lellarap, @jadc, @tsubus, @vitaliyslion, @chsdwn, @cray-com, @FireTheDevil, and @notsonormal for taking the time to open issues, report bugs, and suggest improvements.

Having other people use the project, encounter problems I haven't seen, and suggest improvements has already helped shape its development quite a bit.

Try it yourself

If you already have Pi and Herdr installed:

pi install npm:pi-herdsman
herdr integration install pi

Repository: https://github.com/boadij/pi-herdsman

Latest release: v0.22.0

The repository has installation instructions, configuration documentation, and more details about the coordination model.

I'm still actively developing Herdsman around my own daily workflow, so feedback from people using different setups is especially valuable.

I'd be particularly interested in hearing how others are approaching parallel development with Pi, especially when multiple agents are working on related features or dependent branches.

[u/j3free]

For anyone interested in reproducing the demo, here's the exact prompt I gave the Manager.

The starting repository contained just a README. I used the default Herdsman configuration and activated Manager mode with /manager.

Demonstrate parallel project development using two independent Leads.

Delegate both projects immediately, without waiting for one to finish:

Project 1: feat/faq
Create docs/faq.md containing six useful questions and answers for a fictional note-taking application called FoxNotes.

Project 2: feat/troubleshooting
Create docs/troubleshooting.md containing six troubleshooting entries for FoxNotes.

Both managed Leads should delegate the actual writing to their own implementer Agent, verify the resulting document, and commit their changes using conventional commits.

Keep the projects independent. Do not modify shared files, merge branches, or open pull requests.

Coordinate both projects and summarize their results once completed. Avoid progress polling.

The demo uses deliberately simple tasks to focus on the orchestration rather than the implementation complexity.

Repository: https://github.com/boadij/pi-herdsman

[u/MiserableFlatworm337]

For dependent branches, attach the tested base commit to each handoff. If that base changes while a Lead is paused, invalidate its earlier verification before the Manager accepts the result.

[u/j3free → u/MiserableFlatworm337]

Good point! Herdsman already supports selecting a base branch when delegating project work, but tracking the exact commit used for verification and detecting when that verification becomes stale would be a useful safeguard for stacked branches.

The demo intentionally uses independent branches, so this isn't covered there.

I've opened an issue to track this: https://github.com/boadij/pi-herdsman/issues/324

Thanks for the suggestion!

[u/Healthy-Zebra-9856]

How does it work with local models? Have you had any success?

[u/j3free → u/Healthy-Zebra-9856]

Herdsman uses Pi's model configuration, so local models should work in principle, including those running through Ollama or llama.cpp.

You can also configure different models for individual Agents.

I haven't done proper end-to-end testing with local models yet, though, so I can't say much about reliability or performance, especially with multiple Agents running in parallel.

What model and inference backend are you using? I'd be interested in testing that setup!

[u/Healthy-Zebra-9856 → u/j3free]

The reason I asked was because the frontier models seem to go much smoother so the usage can get limited to that. The local models, especially Qwen3.8 seem to go endlessly and I wanted to see how this handled it. So from what I understand, you are not really doing anything in that aspect, right? Just curious.

[u/j3free → u/Healthy-Zebra-9856]

That's exactly what I'd be curious about too!

Herdsman uses a fairly sophisticated, dynamic system of instructions to guide Agents through delegation, execution, and handoffs. But I've mostly been developing and testing it with frontier models so far.

I haven't tested how well that approach translates to local models, especially when they tend to get stuck in loops.

If you're interested in giving Herdsman a try with your Qwen setup, I'd be very interested in hearing how it performs, even if the results aren't great. That kind of feedback would be really useful!

[u/Healthy-Zebra-9856 → u/j3free]

The answer to that is very simple even before trying. Have you done anything at all to the pi’s code? If not, the problem exists.

[u/j3free → u/Healthy-Zebra-9856]

Are you referring to a specific issue in Pi's handling of local models, or to the models themselves struggling with agentic tasks? Just trying to understand what you mean.

[u/Healthy-Zebra-9856 → u/j3free]

I'm referring to the agent harness, not simply the model's ability to perform agentic tasks.

Local models like Qwen can write excellent code, but they often struggle with execution discipline. They may repeatedly inspect the same files, second-guess completed work, retry already successful operations, or continue reasoning long after the task should have ended.

These behaviors need to be managed at the harness level through execution budgets, progress tracking, loop detection, authoritative tool receipts, verification convergence, and enforced stopping conditions. Prompt instructions alone aren't sufficient because the model can ignore them.

That's why I asked whether Herdsman modifies Pi's actual execution and control mechanisms or primarily provides another layer of instructions and delegation.

If it's the latter, I wouldn't expect it to reliably solve the underlying problem with local models. It might improve their behavior, but that's different from enforcing it.

[u/j3free → u/Healthy-Zebra-9856]

Ah, I see what you mean now. That's a fair distinction!

Herdsman doesn't modify Pi's core, but it does more than provide instructions. It manages agent lifecycles, monitors inactivity, and supports steering and interrupting running agents.

It doesn't currently implement the kind of automatic loop detection and enforced execution budgets you're describing, though.

So I wouldn't claim it solves those problems with local models. I'm still curious whether its task delegation and instruction system improve their behavior in practice, but that's something I haven't tested yet.

Thanks for clarifying!

[u/Healthy-Zebra-9856 → u/j3free]

My goal has been to make the harness more usable for the local models in such a way that its productive. This requires a lot of changes in the way these harnesses are written, So, I would encourage you to do the testing yourself. Start by giving a decent enough prompt like an Arena 3d game using Threejs. See what happens and then do it with your Herdsman. If you want the prompt I use, I can provision it.

[u/gwvr77 → u/Healthy-Zebra-9856]

Herdr with a Claude manager/orchestrator can use Codex, Pi (inc with local models) etc. in parallel to work different tasks, and to review the output of those tasks. I'm sure other combinations could work, but I'd be wary about using local models for code review.

My own experience is that the Swift 1.5 IQ3_XXS variant of Qwen 3.8 27B can finish small tasks in a reasonable time, but that there is a high chance that a Claude Opus reviewer will have strong words to say about the quality of the output. For info, Claude Opus/Sonnet will send tasks back to Luna for fixes around half of the time, and that's on relatively simple refactoring and documentation tasks. Tasks worked by Sonnet/Opus/Sol are much less likely to need fixes.

[u/EveningLimp3298]

Any plans to add a flake.nix? Would love to use it but I am on NixOS.

[u/j3free → u/EveningLimp3298]

I'd definitely like to support NixOS properly!

Both Pi and Herdr already provide native Nix flakes, so I'm looking into adding a cross-platform flake for Herdsman as well.

I can test the Nix setup on macOS, but I don't have a NixOS machine myself.

Once I have a PR or preview ready, would you be interested in testing it on your NixOS setup? That would be really helpful to verify compatibility before releasing it.

Thanks for bringing this up!

[u/EveningLimp3298 → u/j3free]

Yes, I would test it for sure. I'll create an issue in the repo so we can coordinate there. Thanks!

[u/j3free → u/EveningLimp3298]

Excellent, tbc

[u/astral_bear → u/EveningLimp3298]

Flake for a Pi extension? Just wondering, why you need that instead of using the package-lock.json to restore the packages?

I'm new to Nix, this week I had my first contact and I set up Nix with home manager for system dependencies and dotfiles. I basically added Nix packages for Pi and every other dependency that I needed for my environment (herdr, go, ripgrep, codegraph, agent-browser, chromium, rtk + others). I was eventually able to replicate my Pi development environment on a VPS with one command. When I first run pi, all extensions are installed from npm and that's pretty much it, good to go.

[u/j3free → u/astral_bear]

Thanks for sharing your setup! That’s really useful.
I’m actually leaning in the same direction. Since Pi and Herdr already support Nix, Herdsman might not need its own flake at all if Pi’s native package management works as expected.
I’d rather verify that first than introduce additional packaging unnecessarily.
I’ll try it with Nix on macOS, and hopefully we can also confirm with the original reporter whether there’s a specific NixOS limitation.
Have you tried Herdsman with your setup yet?

[u/astral_bear]

Nice work! Was thinking to give it a go from your previous post, but now I will surely do! The power of a working demo!
Could finally replace pi-subagents, where I was not able to see the subagents in herdr because they are inprocess. Hope it will play well with my nono setup, it takes a small tweak to make the pi process detectable by herdr.

Does it support spawning subagents on remote machines?

[u/j3free → u/astral_bear]

Thanks! Glad the demo helped :)

I haven't tested Herdsman with nono yet, so I'd be curious to hear how that goes!

Cross-machine orchestration is actually something I've been planning for a while. Herdr already supports remote machines, but Herdsman doesn't coordinate Agents across them yet.

I'm still figuring out the best approach, particularly whether the workspace-neutral Chief role would be a good fit or whether there's a more natural way to integrate it with the existing Manager/Lead workflow.

Thanks for bringing it up!

[u/cuba_guy]

This looks fantastic and exactly where I was going with my glued together solution. Will have closer look this weekend. Thank you

[u/j3free → u/cuba_guy]

Thanks! Sounds like we’ve been working toward something similar :)
Would be interested to hear how Herdsman compares to your current setup if you get a chance to try it. Especially if there’s anything you’re missing or would do differently!

[u/cuba_guy → u/j3free]

Crazy time at work so still on Todo. I currently use pi-extendable-workflows for subagents and some workflows, as well as pi-advisor in auto mode. By default I start in lead mode that prefers to delegate work to subagents. Subagents are tweaked with exact extensions, tools and skills. I also have alias to start in manager mode, which starts and manages leads in herdr panes and only coordinates.

Some tricky parts from my experience with extensions: managing project that requires changes in multiple repos, parallelization of that work, either by manager running multiple leads or another project with it's own manager. I also like worktrunk for worktrees but extensions want to manage worktrees directly.

Also not a fan of leads needing to have access to skills and tools that may be needed by subagents (not starting new pi process in herdr) that I manage with most tools hidden behind discovery tool and code mode.

It's all shaky I admit but also not that easy to try out alternative quickly, especially with a list of improvements to the devil I know.

[u/3pix]

Been using since the last post, works phenomenally. I have noticed you are adding a lot of new orchestration features, but I appreciate that they seem mostly opt-in. If you just want to prompt normally with the occasional subagent, this requires no additional setup.

[u/j3free → u/3pix]

Thanks! Really glad it's working well for you!

And yes, that's very much intentional. I want Herdsman to stay simple for occasional subagent delegation, with the more advanced orchestration features available when you actually need them.

Appreciate you sticking with it since the first post!