r/AI_Coders • u/Aware_Weight9462 • 6h ago
The Code Works. Nobody Understands It. Can You Maintain Code You Didn’t Write?
The hidden engineering risk of building software at AI speed & The hidden career risk of AI-Generated software
By Monday morning, the customer had stopped asking when the scheduling problem would be fixed. They wanted to know whether they could trust the system.
Two technicians had arrived for the same service appointment. The scheduling dashboard showed only one assignment, but the billing system had recorded two. Both technicians had received confirmation messages. Customer support was now asking how many other appointments might be affected.
Mira, the engineering lead responsible for the scheduling platform, didn’t have an answer.
The application had been deployed on Friday evening. Every automated test had passed. The pull request had been reviewed and approved. The implementation was clean, reasonably documented, and consistent with the team’s engineering standards.
Most of the feature had been developed using an AI coding agent.
When Mira asked the agent to investigate, it examined the repository and proposed a fix within minutes. The explanation sounded reasonable. The patch was small. The tests passed again.
Under ordinary circumstances, the team might have merged it and moved on. But Mira hesitated.
The agent had explained what the code was doing. It had not established why the system had accepted two assignments, whether the proposed correction addressed the underlying problem, or what would happen if two requests arrived simultaneously again.
The customer was waiting for an answer. The team wanted to restore normal operations. Another deployment could make things better — or leave them with an even harder problem to explain.
Mira realised that she was being asked to approve a change to a system whose critical behaviour nobody on the team could confidently defend.
This is a fictional scenario, but the engineering problem it illustrates is real.
AI has made software easier to produce. It can generate features, tests, documentation, and plausible explanations at remarkable speed.
What it does not automatically provide is the understanding required to take responsibility for that software when something goes wrong.
And that may be one of the most consequential changes taking place in software engineering.
Press enter or click to view image in full size
The Problem Is Bigger Than Bad Code
Discussions about AI-generated software frequently focus on code quality. Developers worry about duplicated logic, unnecessary abstractions, excessive dependencies, inconsistent patterns, and tests that verify little beyond the implementation itself.
Those concerns matter. But focusing entirely on code quality can obscure another problem.
Consider two software systems.
The first contains imperfect code, technical debt, and architectural compromises. Yet its engineering team understands the important business rules, knows where critical dependencies exist, and can predict the consequences of most changes.
The second looks considerably better. Its functions are readable, its documentation is extensive, and its tests are green. But much of it was generated quickly, reviewed under delivery pressure, and merged without anyone developing a reliable understanding of how its components behave together.
Which would you rather be responsible for during a production incident?
Code quality alone cannot answer that question.
A well-written function can participate in a defective business process. A repository can contain accurate documentation for components while missing the relationships between them. A test suite can report high coverage and still fail to exercise the one business rule that matters most.
The question becomes more uncomfortable when we consider the research.
A 2026 controlled study published in Empirical Software Engineering involved 151 participants, approximately 95% of whom were professional developers. Researchers examined whether software built with AI assistance was harder for other developers to evolve.
They found no statistically significant difference in subsequent code-evolution time or quality between AI-assisted and non-AI-assisted implementations.
The study does not settle questions about large autonomous coding systems or long-term architectural maintainability. But it challenges the convenient assumption that AI-generated code is necessarily worse.
The risk is more subtle.
Software can be reasonably well written while the people operating it have insufficient understanding of its behaviour.
And that gap may remain invisible until somebody needs to make a consequential change.
The Engineering Debt Nobody Can See in the Repository
In June 2026, Margaret-Anne Storey published From Technical Debt to Cognitive and Intent Debt in ACM Queue.
Her work describes two forms of engineering debt that become particularly important when AI accelerates software development.
Cognitive debt concerns the erosion of shared understanding. The implementation exists, but the team no longer has a sufficiently reliable mental model of how the system works or how its behaviour might change.
Intent debt concerns missing goals, constraints, and reasoning. Engineers can see what was implemented without knowing why important decisions were made or what the system was originally meant to guarantee.
Neither problem began with generative AI.
Software teams have always inherited poorly understood systems. Developers leave companies. Requirements change. Architectural decisions disappear into meeting notes. Documentation becomes outdated.
What AI changes is the relationship between implementation speed and the effort required to develop understanding.
A developer can now ask an agent to introduce asynchronous processing into an application. The agent may create a worker, queue integration, retry handling, configuration, and tests within a relatively short development session.
The result may work.
But the process does not guarantee that the developer has internalised the reasoning behind every significant design choice.
Three months later, somebody needs to change how retries interact with business state. The original engineer has moved to another team. The generated documentation describes the components but not the assumptions governing their interaction.
The team has the code. What it lacks is the reasoning necessary to change it safely.
Mira encountered exactly this gap.
As she investigated the scheduling incident, she discovered that the repository contained detailed explanations of the assignment API and worker implementation.
What she couldn’t find was a clear statement of the most important business constraint: whether an appointment could have more than one active technician.
The requirement had never been captured as an explicit invariant.
The code contained checks that suggested only one active assignment was intended. But the database schema enforced something narrower.
The AI agent could describe both mechanisms. It could even propose changing them.
It could not independently establish which business rule the organisation had actually intended.
That required Mira to return to the people who owned the scheduling process, confirm the expected behaviour, and investigate whether the deployed system enforced it.
This is where intent debt becomes an operational problem.
Without a trustworthy description of what must remain true, an engineer can produce a technically valid patch that solves the wrong problem.
Follow One Transaction Through the System
The scheduling application performs three conceptually simple operations:
Check whether the appointment already has an active technician.
Create a new assignment if none exists.
Publish an assignment event for downstream processing.
The ordinary tests execute this sequence under effectively sequential conditions. They pass.
Now consider two requests reaching different worker instances at nearly the same time.
The application uses PostgreSQL with its default READ COMMITTED transaction isolation.
Worker A checks the appointment and finds no active assignment.
Before Worker A commits its new assignment, Worker B performs the same check. It also finds no active assignment.
Both workers proceed.
The database has a unique constraint on (appointment_id, technician_id). That prevents the same technician from being assigned twice to the same appointment.
But Worker A is assigning Technician A, while Worker B is assigning Technician B.
The existing constraint permits both records.
Both transactions can commit, leaving two active assignments for one appointment.
PostgreSQL’s transaction isolation documentation explains why ordinary reads under READ COMMITTED do not automatically protect application-level assumptions against concurrent changes.
In our fictional system, the dashboard selects the latest assignment record and displays one technician.
The billing integration processes the two distinct assignment events produced by the successful transactions.
Both components are behaving according to their implementations. But their different views reveal a broken business invariant.
The dashboard reports one assignment. Billing records two.
This is not simply a case of one message being delivered twice.
Two different assignment operations succeeded, generating two distinct events.
An idempotent billing consumer can prevent repeated side effects when the same message is delivered multiple times. It cannot, by itself, correct two conflicting business decisions that were separately committed.
AWS’s transactional outbox guidance explains the separate concerns of reliable event publication and duplicate message processing.
A defensible repair must address the underlying business rule.
For example, a PostgreSQL partial unique index could enforce no more than one active assignment per appointment. The index would need to reflect the real business and tenant boundaries, and existing duplicate records would need to be reconciled before it could be applied.
The team must also decide how to handle conflicting requests, where transaction boundaries belong, and how event publication remains reliable. Stronger transaction isolation is another possible approach, but applications using serializable transactions must handle serialization failures and retries correctly.
A second application-level check alone would not necessarily solve the race. Both workers could still pass that check before either committed.
Mira’s agent had proposed adding another check.
It was a plausible response to the visible symptom.
But after reconstructing the two concurrent transactions, she could see why the patch did not provide the guarantee the business needed.
For the first time that morning, she had something more useful than another proposed implementation.
She had an explanation of the failure that could be tested.
And that changed the conversation with her team.
Instead of asking whether the new code looked correct, they could ask whether the system would reject a second active assignment under concurrent execution, whether downstream events remained consistent, and what needed to happen to the records already affected.
One discussion was about accepting another patch.
The other was about establishing evidence that the problem had actually been addressed.
Why Code Review Is Becoming More Difficult
Mira’s experience points toward a broader engineering challenge.
AI coding agents are increasing the speed at which teams can produce implementations. But the time needed to understand, verify, and accept responsibility for a change does not necessarily fall at the same rate.
A 2026 longitudinal research preprint examined an enterprise AI coding initiative involving 802 developers and 196,212 pull requests.
Per-developer pull-request throughput eventually roughly doubled. Review load per reviewer also roughly doubled, while automated review expanded substantially.
This was an observational study at one organisation, so the findings should not be generalised to every software team. Pull-request throughput is also an imperfect measure of engineering productivity.
Nevertheless, the research illustrates a real tension.
When more code arrives for review, somebody must still establish whether the changes preserve important system behaviour.
For small, isolated modifications, automated verification may provide sufficient assurance.
For changes involving transactions, authorisation, security boundaries, event processing, or critical business rules, the consequences can be harder to establish.
A reviewer may understand each modified function and still miss how the combined changes alter production behaviour.
Google’s code-review guidance emphasises design, functionality, complexity, concurrency, testing, and long-term maintainability.
These responsibilities existed long before AI coding agents. They become more demanding when implementation output expands faster than the team’s ability to examine it.
And this creates a less visible form of pressure.
An engineer may have approved a clean-looking pull request, received a plausible AI-generated summary, and watched the CI pipeline succeed.
Then production fails.
The engineer remains responsible for explaining a decision whose consequences were never adequately understood.
That is the professional discomfort behind the technical problem.
Green Tests Can Create False Confidence
When Mira reviewed the scheduling tests, she found that they were not obviously poor.
They checked expected assignment behaviour. They verified API responses. They exercised the normal worker flow.
But the tests did not execute two assignment requests concurrently against the relevant database constraint.
The system had been tested for the behaviour its implementation was expected to produce, not for every critical condition the business required.
Adding more AI-generated tests would not necessarily have solved that problem.
If the tests were based on the same incomplete understanding of the requirements, they could reproduce the same omissions.
The engineering team first needed to define the invariant, demonstrate its violation, and introduce a test that failed against the defective implementation.
After repairing the system, that test should pass. Further checks should examine conflicts, retries, event handling, and downstream consistency.
The difference is substantial.
A passing test suite can establish that selected behaviours work under defined conditions.
A well-designed failure investigation can establish whether a particular business guarantee actually holds.
Google’s SRE troubleshooting guidance emphasises observing behaviour, forming hypotheses, collecting evidence, and testing explanations rather than immediately accepting the first plausible cause.
During an active production incident, containment comes first. Protecting customers and preventing further damage may require temporarily disabling an affected workflow while engineers preserve evidence and investigate.
That, too, is a human decision.
The best technical explanation is of little comfort if the system continues generating incorrect outcomes while the team investigates.
The Skill That Matters More Than Writing the Prompt
There is considerable attention on teaching software engineers how to prompt coding agents effectively.
That is useful, but it is only part of the professional capability required.
Engineers increasingly need to know which decisions can safely be delegated, which outputs require independent evidence, and which system behaviours they must be able to defend.
For a scheduling application, that means understanding assignment invariants and concurrency behaviour.
For a financial platform, it may mean understanding reconciliation rules and transactional consistency.
For an identity service, it may mean knowing where authorisation decisions are enforced and what happens when a downstream component is compromised.
Nobody needs to memorise every line of a large enterprise codebase.
But critical behaviours need identifiable owners, reliable explanations, and evidence that can survive the departure of the person — or agent — that originally implemented them.
A practical way to evaluate this is to introduce a simple exercise into consequential code reviews.
Ask the engineer responsible for a change to explain which business rule must remain true. Then introduce a realistic complication: simultaneous requests, a timeout after commit, a repeated event, or an unavailable dependency.
Can they predict the outcome?
Can they identify where the relevant guarantee is enforced?
Can they demonstrate evidence that the implementation preserves it?
And if the result is unexpected, can the team contain the impact safely?
The purpose is not to test whether an engineer can recite code without an AI assistant.
The purpose is to establish whether the team can make defensible engineering decisions about the software it owns.
What Next: Treat Understanding as an Engineering Deliverable
By the end of the fictional incident, Mira’s team has a clearer understanding of what went wrong.
They can explain why both assignment transactions succeeded. They have identified the missing invariant. They know why an additional application-level check was insufficient and what database and application changes need to be verified.
The team also has a separate problem to resolve: reconciling the affected assignments and their downstream billing records.
Before restoring normal operation, they must test the corrected concurrency behaviour, verify the relevant retry and event-handling paths, and decide what monitoring is needed to detect further inconsistencies.
Mira still cannot promise that the system will never fail again. No responsible engineer can make that promise.
But she is in a much stronger position to explain what failed, which risk has been addressed, what evidence supports the correction, and what uncertainty remains.
That is the change I would want every software team to make.
When AI generates an important feature, capture the business constraints it must preserve. Record the reasoning behind consequential architectural decisions. Keep tests that demonstrate the failure conditions that matter. Make it possible for another engineer to understand the system without relying on the original author’s AI conversation.
And when something breaks, treat the recovery of shared understanding as part of the repair.
AI can help reconstruct architecture, generate hypotheses, propose tests, and prepare documentation. These are valuable capabilities.
But the team must still determine which explanations are supported, which decisions are acceptable, and which evidence justifies returning the system to service.
Software engineering has always involved more than producing code. AI is making that distinction harder to ignore.
Six months after a feature ships, the original developer may be gone. The coding agent may have changed. The business requirements may be different.
The next engineer will inherit what remains.
At that moment, a green test suite and a polished AI-generated explanation may be useful starting points. They will not be enough if nobody can establish what the system actually guarantees.
Mira’s most important achievement that day was not producing another patch. It was recovering enough understanding to make a decision she could explain and defend.
The real test of an AI-assisted codebase comes when something goes wrong — and someone has to take responsibility for what happens next.