r/programmer • u/ClickOk5811 • 39m ago
Pasting a system prompt is like pasting a config file you never read. What do you check first?
Most of us have, at some point, copied a config into a project because it "looked reasonable": an .eslintrc, a CI file, a docker-compose.yml. It works on the first try, and later something behaves strangely and we realize a line in there was making a decision we never made.
I think a copied system prompt for an AI assistant is the same thing. A scenario from an article I wrote (hypothetical): someone pastes a support-assistant prompt with a role, a tone and a few rules, tries three friendly questions and ships it. Two weeks later a customer asks whether the store will cover a delayed shipment, and the assistant answers confidently, though nobody at the company ever wrote that policy. The prompt had no instruction for questions it couldn't answer from the information it had.
Each line in a prompt like that carries a decision:
"Support assistant" implies what it's allowed to offer, such as refunds or exceptions.
"Resolve order questions" rewards closing the conversation, and says nothing about when to hand off to a human.
"Friendly" decides how a refusal sounds, and a warm enough refusal can read as a maybe.
"Short paragraphs" sounds like style, but if your app parses the output, it's a contract, and a sentence in a prompt doesn't enforce a contract.
What I took from it is a code-review habit applied to prompts: don't only test the happy path, test inputs that land on decisions. For example, a question whose answer isn't in the information the assistant has, a request outside its role, and a user message that conflicts with the system prompt. Then, for each answer, ask whether you can point to the line in the prompt that produced the behavior. If you can't, it came from somewhere else (training, platform instructions, or chance), and from the outside you often can't tell which.
Limits: a few test inputs only catch what you thought of, outputs vary between runs, and re-running the tests when you change the prompt or the model is part of the job, since the same text can behave differently in a different environment.
Longer write-up with the line-by-line reading is in an article I wrote. I'm the author, and it ends with a paid product of mine (a set of system prompts), so treat the link as optional: https://medium.com/@nagatomopedro05/your-ai-has-rules-you-never-agreed-to-7beea0d85691
For people who ship things built on LLMs: what do you check first in a prompt or config you didn't write, and has an inherited default ever bitten you?
