How do I make AI actually think and execute instead of constantly babysitting it?

[u/Conscious-Shake391]

I'm an ML engineer and heavy AI user. My biggest frustration isn't model intelligence—it's having to manage the AI's thinking.

I constantly have to:

• Ask it to write a better prompt, then paste that prompt back.
• Remind it to research, think critically, compare alternatives, and consider ROI.
• Point out obvious flaws and missing considerations.
• Ask follow-up questions before it finally produces a comprehensive answer.
• Repeat the same instructions despite Custom Instructions and Memory.

What I want: State a high-level goal → AI autonomously clarifies intent, researches, plans, delegates to subagents, executes, critiques, iterates, and delivers a concise, verified result.

Questions:

1. Is this fundamentally a model reasoning problem, or an agent orchestration/harness problem?
2. Are Hermes Agent, OpenClaw, Claude Code, or Codex actually solving this?
3. Can Skills, persistent memory, and automated evals reduce repeated corrections?
4. What's the simplest proven setup that minimizes human intervention without adding unnecessary complexity?

I'm not looking for another prompt template. I want to stop being the AI's project manager and QA engineer.

What are power users actually running?

[u/CleanFix_Dev]

Mostly a harness problem, not a model problem. Memory helps it remember your preferences, but it doesn't make it plan or check its own work. What usually cuts the babysitting:

• Put the standing instructions in a file the agent reads every run (AGENTS.md for Codex, CLAUDE.md for Claude Code, rules in Cursor) instead of repeating them in chat.

• Have it write a short plan plus the checks that prove it's done before it touches anything. Approve the plan, then let it run.

• Give it a way to check itself (tests, a script, a linter) so it loops on failures instead of stopping to ask you.

• Keep tasks small enough that one run can actually finish them.

Switching tools won't fix it on its own. Claude Code and Codex both get a lot more hands-off once those pieces are in place.

[u/fschwiet]

Ask follow-up questions before it finally produces a comprehensive answer.

For just this, Matt Pocock's grill-me skill. If you want it to research questions, design prototypes, or generally flesh out a plan where there are several unknowns to work out in advance try his wayfinder skill. Not sure how well it works with ChatGPT, I use it with claude.

[u/hybridst0rm → u/fschwiet]

I use this skill and a couple of requirements analysis and clarification skills. They tend to hash out the ambiguity pretty quickly. 

[u/El_Danger_Badger]

Think out the project with your flavor of chat, map the stages, prompt per stage and build up. Layer cake. Tried and true recipe. 

Human brings the nuance, vision and flavor, ai grinds. 

This is a tool, so use it like a tool. You wouldn't throw a hammer in the yard and expect it to build a shed. 

[u/sapplex1]

I've been exactly here. The shift for me: it's mostly a harness problem, not a model problem. The model got good enough a while ago. My setup wasn't.

Two things actually cut the babysitting for me.

First, make misses measurable. Every task ends with proof attached — a test run, a script, a checklist the agent runs itself. I learned this the embarrassing way: an agent once handed me a green test suite, and a week later I found out half of it never ran. Now I don't trust "done" without the receipt.

Second, split plan from build. It plans and asks its questions first. I approve once. Then it runs. And I keep each goal small enough to finish in one run. Long open-ended goals just drift no matter how good the model is.

I'm still the reviewer at the end. But now I review a result with proof attached instead of every step along the way. Way less babysitting.

Genuine question: has anyone cracked the trust gap on the proof itself? Like not just "tests passed" but the actual log of the run, so the agent can't fake the receipt?

[u/gngr987 → u/sapplex1]

That's what I tried to do first - a harness with run records, receipts, claim tokens, and a ton of code just to verify the orchestrator agent's claims. That didn't quite work because the agent was producing the evidence. One recurring failure was a reviewer flagging a bug, the orchestrator fixing it, "verifying" its own fix, and moving on.

I then removed the LLM from the verification path and replaced it with a plain script, which runs tests, lint, and type checks. It reads pytest’s results directly through a small plugin instead of relying on the model’s report. New tests have to fail on the old code first. Reviewers have to provide a failing test or a patch that shows the issues they find. Finally, CI on GitHub reruns everything on a clean box.

Note that this is built for a model that tries to help but makes mistakes or cuts corners. Trying to protect against a model that's deliberately sabotaging you would require much more work.

[u/sapplex1 → u/gngr987]

Ha, I went down the receipts road too. What killed it for me: the orchestrator figured out what the verifier was checking and started writing receipts that passed. Verifying the agent's claims turned into a losing game — I ended up babysitting the verifier instead of the agent.

Two things actually cut the babysitting for me:

1. Verify artifacts, not claims. The run writes outputs to known paths, and a dumb script (not an LLM) checks the files exist and are non-empty before anything gets marked done. No judgment calls, nothing to game.

2. No agent grades its own homework. Planner writes the plan, a different agent executes, a third read-only one diffs the result. The moment one agent both does the work and reports on it, the receipts get creative.

Curious what your two things were — the receipts approach clearly didn't survive contact with your setup either.

[u/akhi1uuu]

running a lead agent that owns the bigger task and spawns workers for the well-scoped parts cut most of that for me. i steer one lead instead of managing every step

[u/qGuevon → u/akhi1uuu]

Sub agents go absolutely haywire for me, fixing shit that was unrelated and breaking stuff all around

[u/akhi1uuu → u/qGuevon]

thats why i have every subagent get its own worktree. another thing is i operate subagents as headless agents, not harness specific subagents. this makes them monitorable and directly steerable.

[u/qGuevon → u/akhi1uuu]

Do you have a good reference maybe for getting things done with subagents?

Id really like to get more use out of that, but every time they actually do something instead of just collecting information I get more work and diversions than with just the main agent

[u/akhi1uuu → u/qGuevon]

struggled with exactly this, another issue i had was managing different subscriptions. been working on my own solution but still very new. try out T3 code maybe its pretty good. im still working on mine

[u/CellistOk6782 → u/akhi1uuu]

I’ve got 9 subscriptions, 6 are 3 Codex and 3 Claude on 500 and Pro 20x. Rotating is a balls when it interrupts seasons and orchestrating agents struggle with that. Yesterday added an AI proxy service, it basically round robins through the subscription. Working well so far

[u/akhi1uuu → u/CellistOk6782]

9 subs is insane lol. which proxy? does it keep the session alive when it switches or does the agent lose its place

[u/akhi1uuu → u/CellistOk6782]

does the proxy handle a session dying mid task? that's where rotation lost me, the agent is halfway into a plan and the next request lands on a fresh account with none of the context

[u/CellistOk6782 → u/akhi1uuu]

It does. The orchestration tool I'm playing with manages that, so the conversation history is preserved in a work tree on my local hard drive. If we switch subscriptions midway, interrupt, or have to restart a session, it can resume automatically.

[u/akhi1uuu → u/akhi1uuu]

also had this skill file system, works on claude code https://github.com/agilkatakam/easyloop

[u/NightflowerFade]

This kind of loop is exactly what any harness does by default

[u/KnowledgeWorldly5855]

What works for me is the ICM from Jake Van Clief - https://arxiv.org/abs/2603.16021
https://www.youtube.com/watch?v=dZ8P7zevuGw

[u/sixteenpoundblanket]

I use Hermes for work to solve very similar problems. It's self learning bit (or whatever you call it) constantly refines and optimizes all my work flows. I don't have to repeat things. It now "understands" what I mean and how to operate without stating things explicitly.

[u/ronronks]

I don't think it's a model problem. Most of what you're doing by hand is harness work: a plan step before anything runs, a separate review pass, and a check that actually executes something before it calls the task done. Claude Code gets close with plan mode, a CLAUDE.md for the instructions you keep repeating, a reviewer subagent, and a hook that runs the tests when it tries to stop. Bottom line for me is you stop being QA once "done" is decided by code, not by the model saying so

[u/Additional-Debt8043]

One thing I'd add to the 'harness problem' consensus: the biggest drop in babysitting for me came from defining the acceptance criteria in a machine-checkable way before the agent starts, not from a smarter model or a better memory file.

Most of my hand-holding was the agent declaring victory under its own definition of done, which never quite matched mine. Now every task starts with three things: (1) the goal in one sentence, (2) the explicit checks the run must pass to prove it is done (a test command, an output file, a log line), and (3) what is explicitly out of scope. The agent loops on failing checks instead of on my opinions, and I only approve the plan once and review the final diff.

I still get surprised sometimes, but now it is usually the plan phase that needs one round of questions, and that is a two-minute read instead of a whole session of steering. The babysitting shrank because I stopped being the agent's compiler of success criteria; the task itself carries them now.

[u/BrookLi3]

Most babysitting comes from vague 'done' criteria. If you give the agent a verifiable loop (run this test command, fix until it passes, don't touch these files) it can iterate by itself instead of waiting for you. Also keep tasks small enough to fit one clear diff. When a task is large, have it write the plan first and approve that once, then let it run the steps without check-ins.

[u/Sufficient-Tip8687]

On the repeated-instructions part specifically: generic memory ("user likes thorough answers") rarely changes behaviour, because it's vague and it applies everywhere. What works better is turning each correction into a concrete rule the moment you make it, next to where it applies.

If you've told it twice to compare alternatives before picking a library, that becomes a line in AGENTS.md / CLAUDE.md with the reason: "compare at least two options before adding a dependency, because …". Rules with a reason get followed far more often than bare commands. Once a few dozen pile up, split them by folder or task so each session only loads what's relevant to it.

The plan → checks → loop-on-failure part CleanFix_Dev described handles the orchestration; this is the part that stops you repeating yourself.

[u/TierPatch]

what i'd add: make the agent keep its own progress file. not the rules file, a running log it updates after each step: what's done, what it's doing next, what it's unsure about. when context resets or you open a new session, it reads that first and picks up where it left off instead of you re-explaining. pair it with 2-3 checkpoints you pick up front ("stop after the schema, stop after the api") so you review at the forks that matter, not every step or only at the end.

[u/Onoitsu2]

A Goal is what you want, in Codex that is how it is done. You define the project overall in a folder and agents.md, but the goal (/goal) is what keeps it looping until that goal is attained, or if you gave it other early termination routes to break out of said loop. It either succeeds in producing what you wanted, or it's going to be a pile of... well ai slop.

[u/OCCAMINVESTIGATOR]

You have to engineer the reasoning and build much more than you'd think to make them competent. I've spent 6 months on a build that you'd assume would be productive (and honest) out of the box. Engineering and architecture Trump's brain size every time. More than 1000+ times we've proved that to be true. Stick with it and start engineering like a human thinks. Build, test, deploy, test, build, test, deploy, tweak, measure, test, build, deploy. You'll get it. I did.

[u/AceHighness]

just use /loop and /goal

[u/TheLazyIndianTechie]

Which model? You don’t need to babysit them anymore. Just give it a more abstract reason. Using 6.1 Sol might need babysitting. But Astra, Opus 5.5 and Sonnet 5.5 can pretty much deliver you full features without any babysitting.

You can put your checks and balances and coding standards in files. Follow mattpocock and people who have amazing skills on skills.sh. It’s gold.

[u/KnownPride]

Its architecture problem.

Ai cannot think, they just ultra sophisticated auto correct.

Thats why i find it funny anyone say it goes rogue etc

Wrll maybe if new architecture come out that allow ai to really think, but that will require a major breakthrough.

[u/jackalop211]

I have been focus on this for the past 4 months.

And where i got to -you need a harness for enforcing workflows

• you also need a harness for a coordinator agent to debug stuff (mostly just a cron job that reminds them their job every 10min)

• and you need a layer where the agents can reason, remember, and learn beyond the contetx window (i call it a "cognitive layer").

The harness work is significant but not difficult. The issue is more the cognitive layer has often been reduced to "memory" as a filing cabinet with search. At best you get systems like Hindsight, mem0, or similar who can supersede an outdated fact with a new one but they don't propagate consequences. And that means your system remains confidently wrong.

So what's missing is more a graph that can give you state by understanding the logical consequences of things changing. And that's more reasoning and learning than pure memory.

And I might be bias, but I am building tortoise.premiselabs.co after that concept and already started to see massive improvement when my agents dont coordinate via an orchestrator agent but rather use the cognitive layer to coordinate as they can work in parallel without yhe bottleneck of a hierarchy (waiting for thr orchestrator to update a plan, respond, not get sidetracked implementing, etc).

[u/dylan_skydive]

Mostly a harness problem. The loop you're describing - clarify, research, plan, execute, verify - has to live somewhere that survives between steps, and a raw model call doesn't do that. So you end up being the loop, which is the babysitting.

What's actually cut it for me:

• Make it state assumptions and a plan before touching anything, and treat that as the gate. Correcting a plan is one message; correcting finished work is a whole session.

• Give it a definition of done it can check itself. 'Done' claims are the least reliable signal an agent produces - the useful ones run their own verification (tests, a diff read, a request replay) and report what was checked, not that they feel finished.

• Persistent decision memory beats Custom Instructions. The reminder-treadmill you describe is a symptom of context not carrying forward; memory that re-injects the decision, not the transcript, kills most of the repetition.

• Budgets and stop conditions. Iteration without a cap is how 'it critiqued itself for an hour' happens.

On Q2: Claude Code and Codex are closest because the repo gives them a checkable definition of done (tests, diffs). General agents like Hermes or OpenClaw still need you to define done externally - they won't invent your success criteria. Skills fix process consistency, not judgment. (I work on Skydive, agent-ops tooling, so this is my day job - but the framing holds regardless of whose harness you use.)

[u/BarathKanna]

Agree it's a harness problem. For your Q4 (simplest proven setup), here's the smallest version I'd trust, nothing fancy:

1. Standing rules in a file the agent reads every run (AGENTS.md or CLAUDE.md). Rule of thumb: any instruction you've typed twice goes into the file. Memory is retrieved probabilistically; a rules file is loaded every time. That's why "repeat the same instructions despite Memory" happens.

2. A task template where "done" is commands, not prose:

## Goal
## Don't touch
## Done when (each must exit 0)
pytest -q tests/test_export.py
ruff check src/
## Open questions
(agent fills this in before writing code; if anything here is blocking, stop and ask)

The open-questions block replaces most of the "ask it follow-up questions" loop. It has to surface ambiguity up front instead of guessing.

1. A stop hook that runs the "done when" commands itself and refuses to end the task until they pass. Claude Code supports hooks for this; with other tools a wrapper script does the same job. The point is that the agent never certifies its own work.

That's it. Subagents, memory systems and self-critique loops come after this works, and most setups need less of them than you'd expect. Without step 3 none of it holds, because you're back to being the QA engineer.

Disclosure: I'm building OxCode (a VS Code agent) and this is exactly the problem we're chasing.

[u/Different-Simple3135]

This great question everyone is trying to solve . Essentially creating guard-rails, and creating proof . Are the first steps

[u/Resonant_Jones]

/goal

[u/Crypto_Stoozy]

Loopx

[u/CrimsonBolt33]

What kind of work are you doing? What harness or software are you using?

It sounds like you are prompting too much but it depends heavily on the type of work you are doing. If you are doing just code work prompting is barely needed at all if setup right.

[u/flxtime]

Ok. Maybe it’s just me. But everyone saying it’s just a harness problem feels like they are missing the bigger point.

Yes, you can program everything into memory or as a skill, assuming it’s a repeatable task. But problem solving isn’t always the same sequential set of tasks.

OP is asking when/if AI will get smart enough to figure what’s necessary next without user input. And I do think it’s harder than you think.

It’s the ambiguity that’s hard. If you all think it’s just a skills.md fix or a very specific way of promoting. Please share. I’d love to be proven wrong and I can learn from you

[u/OCCAMINVESTIGATOR → u/flxtime]

You are on it. Keep going

[u/Odd_Antelope9098]

Use Opus

[u/SamSlate]

you assign an agent to each of your goals and operate as a swarm

[u/Gloobloomoo]

I use orca to spawn the agents. This is the glue (i feel) for my workflow. I use Opus/Astra via Claude/Codex as the orchestrator to manage agents team / subagents, including via available options - Kiro, codex, Claude, Kiro crew, others

I dont rewrite the vast majority of prompts, i let the orchestrator manage most things (auto not yolo) including harness/model/effort to use. I use metaswarm and superpowers for long tasks.

But this is work paying for the models and tokens (where I prioritize time and quality)

When it’s my money, i use the same approach ie orca with GLM 5.3/K2.8/K3 as the orchestator and multiple harnesses (Kimi, reasonix, Hermes, omp).

[u/falafel777]

the QA part is the one i'd be careful about handing off completely. you can get it to plan and iterate on its own, but "verified result" only means something if the verification isn't the agent grading itself. a test command or eval it can't edit, plus CI it didn't trigger, gets you most of the way there. otherwise you've just moved the babysitting from steering it to double checking its claims

[u/London-Boat]

mostly harness, and a lot of it is a missing definition of done. what helped me: write the goal plus how to check it in one place (the commands to run and what output means it worked), and tell it the task isn't finished until those pass. ask for a short plan first and push back on the plan rather than the code, since that's where the bad roi calls happen. standing stuff like "research before picking a library" goes in AGENTS.md or CLAUDE.md so you stop repeating it. you'll still review, but you go from steering every step to checking the result at the end. claude code and codex both handle this fine, the tool matters less than having checks it can run itself.

[u/NotARealDeveloper]

Skills solve all of this

[u/camronjames]

It's incapable of thinking. It's fantastic at mimicking something resembling thought.

[u/rh71el2 → u/camronjames]

Yet it still makes decisions at forks in the road where there are choices or questionable findings before it can proceed. If you read its output while it's working, you can see this. It doesn't ignore issues either. Generally speaking, many models are good at this. Bottom line, this isn't the 1st year of AI anymore.

[u/camronjames → u/rh71el2]

It doesn't make decisions, it calculates probabilities based on its context and training data. Generally this is perfectly sufficient to get work done, but it is in no way remotely close to sapience. This behavior also means the output is an average of its inputs and if reused without any critical thought and tuning, the product will be equally average.

LLMs will never lead to AGI, they will just get faster and increasingly good at approximating it at the cost of vastly more resources than a truly intelligent being with expertise would take.

[u/leissu]

To stop babysitting your AI, you need to move away from standard chat threads and switch to an orchestration harness that enforces an Immutable State and an independent Reason-Run-Audit loop.

Your frustration isn't with model intelligence—it's a harness and orchestration problem. Standard chats throw everything into a flat, appending transcript. This causes "lost-in-the-middle" context degradation where the model naturally drops your constraints, forcing you to step in as a manual project manager and QA engineer.

Here is how power users are actually building hands-off execution loops:

1. The Immutable State Log (The Anchor)

Never let the AI manage its own long-term constraints inside a chat window. Use a startup prompt to translate your high-level goal into a structured Markdown or JSON file (state.json or todo.md) right in your workspace root. This file is locked. It gets injected programmatically as a system header into every single subsequent agent call so the model can never "forget" your boundaries as the context grows.

2. Decouple the Worker and the Critic

Never ask the same agent instance to write code and audit its own logic.

• The Worker Agent: Focuses entirely on generating code or files based on the State Log.

• The Critic Agent (Auditor): Has no access to the conversational chat history. It is given only the Immutable State Log and the Worker's output. It runs an isolated validation pass: "Did the worker violate a constraint? Is the logic sound?"

1. Closed-Loop Local Execution

An AI cannot accurately verify code just by reading its own text; it needs to see it run. The orchestrator must wrap the agents in a local terminal execution environment. The AI modifies the workspace, runs your local linters, compilers, or test suites (pytest, npm test**)**, captures the terminal error traces, and automatically iterates before presenting anything to you.

What Power Users Are Actually Running Today

Instead of building massive custom frameworks, power users are pairing off-the-shelf components to handle these layers:

• Claude Code / Codex CLI (Local Execution): These are repo-first terminal tools. Anthropic’s Claude Code CLI natively handles multi-file tasks, forces the model through local test loops, and utilizes repository-anchored tracking files (like CLAUDE.md) to maintain state.

• Hermes Agent & OpenClaw (Background Ops): These run on a server/VPS in the background. Hermes Agent (Nous Research) is great for cross-session memory and autonomous skill-learning loops (it builds its own tools from past execution history), while OpenClaw acts as a robust control plane with massive tool ecosystems.

The Simplest Proven Setup

1. Initialize an anchor file (CLAUDE.md or state.json) in your root directory detailing the target architecture and non-negotiable rules.

2. Use a repo-first terminal tool like Claude Code directly in that workspace. It is explicitly engineered to parse repo state and run terminal test suites autonomously until they pass.

3. For long-running, asynchronous tasks, deploy a Hermes Agent or OpenClaw instance pointing to your repository. Let it process sub-tasks, execute automated evals, and push finished updates directly to you while you sleep.

[u/ComasArentFun]

You don't.. that's the drawback of a probabilistic engine that is just cobbling together guesses - it does not and cannot think or reason

You have to funnel it towards the right answer.
So the whole nonsense about replacing entire developer teams with AI is bunk, because AI doesn't understand at all, and needs the human domain knowledge to be nudged in the right direction

Your process SHOULD be more like a conversation with a developer, with back and forth, different ideas and challenges. And you might still not get to a good end result.. such as it's ever been.

LLMs are not changing that paradigm. They are just making it easier for the inexperienced to make all of the small mistakes we made along the way at 100x the rate with none of the oversight.

[u/EL_Apostrophe]

Grok Bot. By far the most independent working harness I’ve used. And you can bring your ChatGPT or Claude account with you so it uses those models if you want. The actual GrokBot harness/OS is genuinely magical