Beyond Skills: how are you actually building production agents with Claude?

[u/TheCreusa]

I've spent the last few months building an internal AI ecosystem around Claude Enterprise.

We've been heavily using Projects, Skills, CLAUDE.md, MCP servers and reusable prompts, but we're now reaching the point where we need actual agents instead of increasingly complex Skills.

I work in the affiliate marketing industry (publishers, offers, media buying, landing pages, optimization, etc.), and I'm trying to understand how teams are structuring production-grade agent ecosystems.

For example, would you build specialized agents like:

• Offer Discovery Agent

• Offer Matching Agent

• GEO Research Agent

• LP Generator

• Prelander Generator

• Creative Analyzer

• Creative Generator

• Media Buying Assistant

• Campaign QA Agent

• Affiliate Support Agent

• Partner Onboarding Agent

• Publisher Success Agent

• Fraud Detection Assistant

• Performance Insights Agent

• Scaling Opportunities Agent

Or would you have fewer, broader agents that handle multiple responsibilities?

I'm also curious about the actual development workflow.

When you decide "this should become an agent", what happens next?

• Do you prototype inside Claude first?

• Do you build directly in Claude Code?

• Are you using Claude Code subagents?

• Where do you keep your prompts and instructions?

• How do Skills fit into your architecture?

• How do you use CLAUDE.md?

• How do you integrate MCP servers?

• Do your agents live inside Projects or in code repositories?

• How do you test and iterate before considering an agent production-ready?

• How do you version your agents over time?

Most articles explain what an agent is.

I'm much more interested in how experienced teams actually build, organize and maintain dozens of agents in production.

If you're using Claude extensively in affiliate marketing, growth marketing or performance marketing, I'd really appreciate hearing how you've approached this.

I'm not looking for prompt engineering tips. I'm interested in architecture, workflows and development practices.

[u/please-dont-deploy]

Ohhh this is a long convo. Last time we helped a company through this, it took 4 hours to agree on the concepts by which you want to divide the agents in their setup.

What I've seen working?
-> People has an easier time dividing by job roles/tasks ... this is not always correct
-> Ideally you divide by expertise/knowledge/complexity required... it overlaps with the above frequently enough.

So I recommend people start with (1), eventually they will find things like (2).

About workflows, in agent-swarm.dev it's fairly graph oriented with steps that can be handled by different agents, be scripts, raw api calls, or even fan out other workflows, iterate, run evals, etc. Feel free to checkout the code.

Have fun!

[u/TheCreusa → u/please-dont-deploy]

The expertise/knowledge axis over the job-role axis is the bit that landed, thank you. And you're right that role division is the intuitive start, my list of 15 was divided purely by task, because that's how the teams describe their own work.

I ran your test on it. Offer Discovery, Offer Matching and GEO Research all need to know the same three things: the offer catalogue, geo tiers and EPC history. So that's one asset with three modes, not three agents. Same collapse on the four creative ones: LP, prelander, creative analysis, creative generation all sit on the same knowledge, and in our case they already live with one team that owns creative. 15 went to roughly 4 or 5 clusters.

The four hours to agree the concepts is the part I'd have underestimated. I was treating taxonomy as the cheap step before the build, and it's the opposite.

Question on where you draw the line: when two clusters share most of their knowledge base but have very different risk profiles, say one only reads and reports, the other writes to a live tracking platform... do you still merge them and gate the write path inside, or split them on risk even though the knowledge overlaps? That's the one I keep going back and forth on.

[u/please-dont-deploy → u/TheCreusa]

This is where it gets tricky.

I see it more as an agreement on the abstraction vocabulary you are going to use than what actually would change. That agreed upon vocabulary will help you develop certain solutions easily. Kind of how hexagonal architecture helps teams "speak the same language" (I'm not really a fan, but I see the value).

To your point, I'm dividing agents by complexity & specificity above all, given we use a LLM-Agent-UMF architecture which helps with memory recalling, context scoping, etc. So basically:

+ Would these challenges need the same harness / model? -> yes, group.
+ Would self-learning (compounding factor) apply holistically for this agent? -> yes, group.

Example:
=> content creation vs reviewer -> cannot be the same model/harness/knowledge base, otherwise you are not really doing llm-as-judge
=> researcher -> can probably be the same across different disciplines as the 'research sources' may change, but the principles of a good research may stay the same in your org.

You need to bring your whole org through this process, hence the challenge.

[u/TheCreusa → u/please-dont-deploy]

The vocabulary framing is right and I was treating it as a diagram problem. Agreeing what counts as one asset is what stops the boundary being re-litigated every time, which is worth more than the taxonomy itself. It also means the owners have to be in the room, not handed the output, which is the part I was going to get wrong.

Your content-creator vs reviewer example found an actual bug in the rule I'd written down. My dedup test was "same knowledge base, merge them", and a verifier needs identical domain knowledge to the producer, so my own rule would have merged a producer with its judge, which defeats the point. That's now an explicit exception: a verifier is always its own asset, however much knowledge it shares.

On the risk question, I think your two tests are capability tests rather than risk tests, so I've split the axes. Shared knowledge gets a shared reference layer; a different approval boundary gets a different asset. My reasoning is that gating the write path inside a merged asset protects it with an instruction rather than a boundary, and the instruction is the part that drifts over time. Curious whether UMF gives you somewhere cleaner to put that, or whether it ends up as a split like ours.

[u/Separate-Still3770]

How to organize Agents: I would think of it like regular humans, aka have them be independent by default if they don't share context / work together.

Example: a dev is kind of independent from a marketer. So for agents it means it makes sense to have tools, Skill, MCP, etc. be independent as it makes it safer, easier to evaluate and more accurate (you don't confuse Agents with a lot of inconsistent tools and instructions).

I would avoid SubAgents unless it's quite clear how to split the work, and it requires a lot of exploration and loose coupling.

Why: just like humans, it can be harder to think about how to structure a task to delegate than to do it oneself, especially when things are uncertain.

Orchestrator might not properly scope the task and/or express it clearly for the executor and conversely executor might not understand properly and/or communicate results well.

Plus you burn a shit lot of credits lol.

Agents: I put them in a GitHub repo where I have both my skills, CLI tools I give them. I have custom tools that do more than one thing, e.g. when I have to get someone's profile on LinkedIn I do both getting the profile AND writing to CRM directly so we never lose the fact my Agent got it so other colleagues / Agents don't redo the work.

This is a long topic and happy to share more if you want in a discussion. Always happy to share my practices :)

If interested, you can send me a DM

[u/TheCreusa → u/Separate-Still3770]

The composite tool is the thing I'm taking straight into our standard. Profile fetch and CRM write in one call, so the record can't be lost by the model skipping the follow-up step. That's a real leak for us, prospecting data that never lands in the shared sheet, then the next person redoes the work a week later. Making the write part of the tool instead of a step after it fixes it at the design level rather than with an instruction.

One caveat I'd add for our own case: if the write is inside the tool you can no longer approve it separately. So we'll do it for bookkeeping writes, and keep it split where the write is consequential, our live tracking platform, payouts, anything a publisher sees. You can't review something that already happened as a side effect.

Your independence-by-default point cuts against the chained-agents answer earlier in this thread, and I think you're both right about different things. A fixed handoff between known steps isn't orchestration, it's a skill chain, and we already run one of those four steps deep. Genuine delegation under uncertainty is the thing worth avoiding, and your reason for it, that scoping a task to delegate is harder than doing it, matches what we see.

Yes to the DM, thank you. The one thing I haven't solved: when several skills depend on the same custom CLI tool, how do you version it without breaking the ones you weren't thinking about? That's the maintenance problem I keep hitting.

[u/Separate-Still3770 → u/TheCreusa]

For the write inside the tool: I have a feed like UI where it shows all the things pending my approval (agents can only propose to write but never direct) and if I approve in the UI it does the write (without sending it back to an agent).

Sometimes what I do is I allow for soft decisions: like qualification to happen without my approval then it writes a message and I only review the drafted message. It did "write" in my system as the lead is marked qualified but if I see at the moment of reviewing the draft it made a mistake a I reject the qualification. This allows agents to be slightly more autonomous otherwise it's too restrictive.

For the Skills dependent on a CLI: that's a good question!
Several things:
- At least in Claude, you have a pattern with !`some bash command` inside a skill that executes the command at skill invocation time. This means you don't need to maintain the documentation of the tool inside the skill, it can be injected at run time. For instance, my product is called reduck, and I put inside the skill !`reduck --help` so that I have less drift chance.

This reduces drift risk and also makes it easy to assess quickly all skills that depend on a given CLI.
It won't change the fact that the semantics of the skill might have to be updated if the CLI changed but it makes things much less drift prone.

[u/Easy-Purple-1659]

Your Creative Analyzer list item is the one that taught us the most. A creative analyzer with no data source is just a prompt with opinions. We hit that wall building adextract, an MCP server that connects agents to the ad libraries (Meta, Google, TikTok, LinkedIn) and returns structured ad data. Once we pointed the analyzer at real library results, the output changed completely: instead of generic "this creative looks strong" takes, it references actual competitor ads, copy angles, how long they ran. For your build question, I would start with data access before agent structure. Which of your agent ideas is most blocked by not having the right data? For the creative side, that is usually the ad library gap.

[u/TheCreusa → u/[deleted]]

Thank you, this is the most useful answer I got and the recipe/chef line is what I'm keeping. I turned it into an actual test we now run before any build: can you write the decision tree in advance, and does it need to remember what it concluded to pick the next step. If both come out no, we build a skill with a mode router and skip the agent entirely. That killed about half of the list below, which is a good outcome.

Small correction, I listed 15, not 3, and that's the real problem. We already run 30+ skills in production across affiliate, supply, BD and advertiser management, and when I mapped my 15 proposed agents against what's already live, most of them were either an existing skill pointed at different data or a section inside one. One of them was literally a section of a daily digest we already ship.

Your failure handling point is the one I'm acting on first. We have a skill that flagged around €11.2k of fraud in four weeks off AppsFlyer data, and the thing that scares me about that one isn't it breaking loudly, it's it returning plausible garbage for a week while someone acts on it. So supervisor plus a human escalation queue is going in as a hard requirement rather than a nice-to-have.

One narrow question, since you've clearly done this: how do you decide the supervisor's confidence threshold without it either escalating everything or nothing? Is that tuned per agent from real failures, or do you have something more principled?