Storing and Versioning Prompts

Prompts scattered through code will eventually become a blob no one dares touch.

Prompts start as throwaway strings. Six months in, the same prompt is a few hundred muddled words no one dares delete a line from. That's not a skill problem — it's treating a prompt as anything other than an asset.

Symptoms you'll recognize:

• A weird constraint sits in the prompt, no one remembers why, and no one dares remove it.
• The same rule has three phrasings across three features; fixing one misses the other two.
• Production quality suddenly drops, and there's no way to roll back to the previous version of the prompt.

How it rots

The process is almost always the same.

A feature needs a model, an engineer inlines a prompt, it works, it ships. Then it makes up a number; someone adds "don't fabricate data." Then it answers too long; someone adds "keep it under three sentences." Someone else finds the format breaks on certain inputs; two more special cases get bolted on.

Meanwhile another feature needs a model, so an engineer copies the prompt and tweaks it. A third feature copies again.

Six months later: the same "don't fabricate data" rule exists in four different wordings across four files, and one of them never got the rule at all. Someone wants to unify the phrasing and has to hunt through the whole codebase — and after the edit, no one knows whether another feature broke, because there are no test cases.

No single person caused this. It's the classic sequence where every step is reasonable and the accumulation is chaos. The official prompting guides from both major vendors are really about one thing: treat prompts as engineering assets you maintain over time, not one-shot incantations. anthropic-prompt-eng

Three things to do

You don't need fancy tooling; these three practices fix most of the problem.

1. Move prompts out of code. Put them in separate files or config; business code just references them. The immediate payoff: prompt changes get their own diff. You can see exactly which sentence changed instead of digging through a code change.

The bigger win is that shared parts are written once. Rules like "don't fabricate data; say you don't know when you have no basis" are site-wide, defined in one place and referenced everywhere — change once, apply everywhere.

2. Annotate every constraint with its origin. This is the most skipped and the most valuable. For every non-obvious constraint, add a note: what problem it was added to fix, when it happened, and how the test scores moved after.

Why it matters: the most expensive thing in a prompt isn't the sentence itself, it's the knowledge of why it's there. Without it, the next person faces a constraint that looks redundant and can only choose: don't touch it (so the pile grows), or delete it (and the bug from six months ago comes back).

3. Version prompts together with their test cases. The test set from the previous article and the prompt are one unit — keep them together and version them together. The official guides keep insisting on systematic iteration over gut feel, so run the full set on every change; that way you see scores move, and you can trace which version introduced a regression when things break. openai-prompt-eng

If you can, wire the scoring into your regular pipeline so prompt changes run automatically. Even without automation, a manually runnable test set already puts you ahead of most teams.

Who owns it

The last practical question: who maintains the prompt?

It's an awkward object — written like documentation, runs like code, and changing it changes the product experience. So two failure modes appear: engineers say it's a product concern, product says it lives in code so it's engineering, and no one owns it systematically.

The pragmatic split: product owns what the prompt says; engineering owns where it lives and how it's served.

Product decides the background, the constraints, what counts as correct, and the golden answers in the test set. These are product judgments, not technical ones. Engineering handles storage, references, versioning, and wiring scoring into the pipeline.

This split has a bonus: product can edit a prompt and see the effect directly, no sprint backlog needed. Iteration gets an order of magnitude faster — and prompts are exactly the kind of thing that improves through fast iteration.

Here's the test for whether you're managing prompts well: can a new person explain why every sentence is there? If yes, you're in control. If no, it's already rotting — spend half a day cleaning it up before it gets worse.