Managing Prompts

They grow into something nobody dares delete — not a skill problem, but a failure to manage them as assets. Pull them out, record why each rule exists, prune what no longer fires.

Every prompt begins as a scrap of text someone typed in passing. Half a year of product work later it has usually swollen into several hundred words of tangled layers that nobody will touch. This has little to do with anyone's skill; the problem is that it was never treated as an asset.

After reading, you should be able to answer:

• How do you tell that a prompt has already gone bad?
• Once it is out of the code, why is that still not enough?
• What information decides whether a rule should stay?

One question tells you whether it has gone bad

The test needs no tooling: can someone who has just taken it over read it and understand why each sentence is there?

If yes, every rule still carries its own record of origin. If not, those origins are already lost — and whoever added the rule has probably forgotten them too. By that point it has started to decay, and it is worth half a day to sort out early, before you reach the stage where people want to change it and nobody dares.

The test works because the thing of real value in a prompt is not the constraint itself but the record behind it. The next two sections take up how that record gets lost, and how to put it back.

Nobody sets out to wreck it; it piles up

The decline follows much the same script every time.

It starts with one feature that needs the model, so an engineer tucks a block of text into the code. It looks fine, so it ships. Then the first flaw surfaces: it conjured a number out of nothing, so someone adds "do not invent data." Before long the same line reads completely differently in a support thread and an internal ticket, so in goes "no exclamation marks." Later it turns out to be correcting typos in the customer's own words, and two targeted lines get appended.

Meanwhile a second feature also needs the model, so the engineer copies the block over and adjusts it slightly. A third feature does the same.

Look again six months on: the rule "do not invent data" sits in four files in four different phrasings, and one of them never got it at all. Someone wants to unify the wording and has to open file after file; and once the edit is done there is no way to feel confident, because no test samples exist to show whether other features were damaged.

Nobody behaved unprofessionally. This is a classic accumulation failure: every individual step holds up on its own, and the pile of them spins out of control. The official prompting guidance keeps making the same point: a prompt is an engineering asset with a long service life, not copy you write once and never revisit. anthropic-prompt-eng

Getting it out of the code is the first step back to editability

All of that trouble shares one source: the prompt lives in the wrong place. Mixed into the code, its edits are mixed into code edits too, and nobody can see which sentence this particular change touched.

So the first move is to get it out of the code. Store it as its own file or configuration entry, with business code only referencing it. The direct payoff: every edit can be reviewed on its own. Which sentence changed is obvious at a glance, instead of drowning in a pile of code changes.

More importantly, the repeated portions only need writing once. A constraint like "if there is no basis, do not invent data — say there is none" applies across many situations: define it in one place, reference it everywhere, and one edit takes effect throughout.

To judge a rule, you need its origin and a sample

Getting it out of the code only fixes "there is nowhere to make the change." The remaining problem is confidence when you do make one, and that takes two things in place.

One is the origin. For any constraint that is not self-evident, add a note: which gap it was meant to plug, which incident brought it about, and how the sample score shifted after it went in. This is the step most often skipped and the one that pays best. Without that background, whoever inherits the prompt faces a rule that looks surplus and has only two options: leave it alone, so the pile keeps growing; or remove it, and watch the incident from six months ago come back.

The other is the sample. The sample set and the prompt are one unit; they belong in the same place and under the same version control. Run the samples and record the score on every edit, and when something breaks later you can trace which version brought it in. openai-prompt-eng

If you can, hook that scoring into your regular checks so a prompt edit triggers a run automatically. If automation is out of reach for now, that is fine — having a sample set you can run by hand already puts you ahead of most teams.

Growing without pruning is the other half of the maintenance job

The earlier steps handle "nobody dares change it." This one handles "it only ever grows."

The characteristic way a prompt falls apart is growth with no pruning: every incident adds a sentence, and nobody has ever removed one. Over a few years it ends up housing several constraints that fight each other, plus a batch of special cases written for scenarios that went offline long ago. The more items there are, the less weight each one carries — they dilute one another, and the ones that actually matter get buried in the middle.

So pruning is part of maintenance, not an optional extra. To decide whether a rule stays, ask three questions:

• Does the problem it blocks still occur? If the scenario is retired and the entry point is closed, the rule is just an expired record. Delete it.
• Does anything still cover it? If the sample that corresponded to it has been removed or rewritten into something else, the rule has lost its acceptance criterion — a rule with no sample is one nobody can tell is still working.
• Does it fight another rule? Two mutually exclusive constraints coexisting is the most typical product of long-term growth without pruning. The argument to have is not about which one is right, but about returning to the problem each was written to block and seeing which problem still genuinely exists today.

There is one situation that does not justify deletion: you simply cannot tell what the rule does. That only means its origin was never written down. The right move is to add the note and add the sample so it becomes judgeable again — not to delete it and hope for the best.

Product owns what it says, engineering owns where it lives

One practical question remains: who is responsible for the prompt?

Its identity is awkward: it takes the shape of a document, runs like code, and changing it alters the user experience. So two familiar forms of buck-passing appear: engineers decide it is the product team's job, product decides that since it lives in the code it belongs to engineering, and nobody looks after it systematically.

A workable split: product decides what the prompt says; engineering decides how it is stored and invoked.

What it says is product's call — how the context reads, which constraints exist, what counts as good enough, what the accepted answers in the sample set are. Those are product judgments, not technical ones. A prompt is itself a requirements document, and requirements documents have always been product's own work.

Where it lives, how it is referenced, how it enters version control, how scoring plugs into the pipeline — engineering handles those. The split has a side benefit: product can edit it and see the result firsthand without going through scheduling each time. That makes iteration an order of magnitude faster — and prompt quality is precisely the kind of thing that only gets ground out through fast iteration.

The judgment beyond writing

Since a prompt is a requirements document, writing it clearly is the job itself, not a trick: miss a part and it drifts exactly there; examples carry information more precisely than adjectives; whatever a program reads needs its format pinned down; change one thing at a time, with samples to compare against. All of it rests on one precondition — that the prompt itself is in a state where it can be changed, which is what this article is about.

With the asset steady, one variable of stability remains: randomness. Randomness is designed in, not a bug explains why "turning it down" is not "turning it right." Further out, managing the prompt as part of a larger context is what context engineering deals with, and teaching the model things its corpus never contained starts with retrieve, then answer.