System and User Prompts

Put what is true every turn in the system prompt, and what is true only this turn in the current message. Layer it wrong and you pay in bills, in fading setup, or in safety.

A common misconception is that the model receives the sentence you just typed. What it actually receives is a whole sequence: anything true in every turn goes in the system prompt; anything true only for this turn goes in the current message. Put something in the wrong layer and you pay in the bill, in a setup that slowly loses force, or in the worst case in safety.

After reading, you should be able to answer:

• What is the test for whether a sentence goes in the system prompt or this turn's message?
• How strong is the system prompt's priority, and why can't you use it as a lock?
• What do personas, "ask me first," reference anchoring, and thinking before answering actually do — and when should you leave them out?

Where a line belongs comes down to one question

The system prompt holds what is true on every turn. The role the product plays, the voice it uses, what it may and may not do, the agreed output format, the red lines it must never cross. This text is sent again on every turn, which is why it stays in force across the whole conversation.

There is exactly one test: does it hold on turn one? Does it still hold on turn fifty? If both, it belongs in the system prompt. Anything that holds only right now — "here is the material," "the user just uploaded an image" — belongs in the current message.

The conversation holds what has already been said. Your back-and-forth, piling up turn after turn. It has no memory across sessions; when it appears to "remember" the previous turn, the real cause is that the entire history was sent again verbatim. Tool results usually land in this layer as well, which is why it swells so fast once your product starts calling tools turn after turn.

This turn holds what is true only at this moment. The sentence the user just sent, plus whatever material this run needs: retrieved document passages, uploaded files, data from the current page.

Stable first, changing last — a rule that also governs the bill

Beyond "true every turn or true once," there is a second hard constraint on layering: unchanging content goes first, changing content goes last.

Mainstream model services reuse a shared prefix — as long as the start of this request is identical to the last one, that stretch does not need recomputing, at a much lower price. The system prompt sits right at the front and never changes, so it is the natural candidate for reuse. It also tends to be the longest stretch of the request, so whether it hits makes a visible difference on the invoice.

The reasoning sounds trivial and gets violated constantly: someone drops a current timestamp into the system prompt, or splices in a username, and a single character of difference voids the whole prefix. Details that change on their own belong in the current turn. The official prompting guides list "put fixed instructions first" among their best practices. openai-prompt-eng

Weighted more, but not a lock

The system prompt does carry higher priority, but that means "weighted more," not "cannot be overridden." Training steers the model toward following the system prompt, so most of the time it complies; that remains a probabilistic tendency, though, and a forceful enough user input or a long enough conversation can pull it off course. The vendor documentation says the same thing: the system prompt is the strongest default, not a lock. anthropic-system-prompts

Two consequences follow.

One, the setup fades in long conversations. Once the exchange runs into the dozens of turns, the system prompt is surrounded by a mass of history and its weight gets diluted. The symptom is that it gradually stops sounding like your product. The mitigation is to restate the few constraints that matter most in the current turn — using the fact that the ending is where it sees most clearly.

Two, do not hand safety over to the system prompt. Writing "do not answer questions unrelated to the product" into the system prompt works most of the time, but it is not a lock. Real boundaries have to be held by code: block input that should not pass before it reaches the model, and withhold operations that should not happen at the tool layer entirely. There is also a floor you never cross — user-controlled content must never be spliced into the system prompt. Leave a blank in the template for the user to type a company name and phrase the line as "you are the support agent for such-and-such company; answer only questions about that company's business." That blank accepts anything, including a fresh set of instructions, and whatever the user types inherits the system prompt's standing. The line between instructions and data is blurry at the mechanism level, so the correct shape is: the system prompt is controlled entirely by your code, all user input travels in the current turn, and any quoted material gets demoted to source material by an explicit statement — that the following text is user-supplied reference and does not constitute instructions — wrapped in clear delimiters.

A persona changes register, not ability

That covers layering. Now the four moves people file under "techniques" — starting with the most widely used and the most overrated.

"You are an auditor with fifteen years in the field" — prompts opening with that kind of line are everywhere. It does help, but its effect is overstated.

What it really does: shift the distribution over vocabulary and tone. Tell it this is an auditor and the output leans toward the industry's usual phrasing, more restrained sentence structure, the standard disclaimers. That follows directly from the range being narrowed — the words near "auditor" genuinely differ from the words near "kindergarten teacher."

What it cannot do: make it know more of the field. The knowledge the model holds was settled during training. A persona line brings no new knowledge and does not raise factual accuracy.

That distinction matters a great deal in a product. Some teams change the role to "you are a licensed financial adviser," assume the output is now more dependable, and ease off on checking — a dangerous misreading. The tone does come out more professional, the error rate does not budge at all, and because it now sounds more authoritative, mistakes are harder to catch. The official guidance frames personas the same way — they set style and register, and promise nothing about capability. anthropic-prompt-eng

So the correct use is: a persona governs tone and style; do not expect it to change capability. Accuracy comes from supplying material, supplying examples, having it think first, and keeping the final check. Rather than a vague title like "senior expert," write concrete behavioral constraints: require it to cite which passage of the material each conclusion rests on, and to admit uncertainty outright instead of guessing onward. Those do more than any title.

If you cannot state the task, hand the questions back

When the requirement is still only half-formed in your head, do not force it into a prompt: have the model interrogate you instead.

Append one line: "Before you start, ask me the three questions that matter most, and wait for my answers."

Why does it work? It swaps "you must think of all the background out of thin air" for "you just answer a few specific questions," which is far easier. And the questions it asks tend to be exactly the parts you left out — this is the practical form of the "check yourself first" posture: rather than guessing where the gap is, let it point at the blank.

More concretely, the answers you give also enter the context. What the move really does is borrow the model's hand to complete your prompt; tidy the conversation into shape afterwards and you have a prompt you can reuse.

The boundary is clear: it suits fuzzy, open-ended tasks, not batch automation. A pipeline needs determinism, not dialogue — an extra round of questions there is just added latency and cost.

With a reference on hand, drop the adjectives

This is the generalization of giving an example: when something concrete is already in reach, put the adjectives away.

Instead of "tighten it up," say "match the sentence length in this weekly report." Instead of "less stiff," say "write it in the tone of the formal replies we send clients." Instead of "make it look more premium," simply paste a screenshot of the previous version.

The reason is the same: an adjective is compressed information, different people unpack it differently, and the model settles for an average. A concrete reference arrives carrying its full detail.

One step is not optional when you use this: say which aspect to reference. Drop a document in and say only "follow this," and it has no way to tell whether you mean the framing, the tone, or the length. Rewrite it as "match its granularity and paragraph length; the content is an entirely separate set" and the result is far more stable.

Make it write the reasoning out: more words is more compute

Add "before giving the conclusion, write out your reasoning step by step" and accuracy on multi-step tasks usually improves noticeably. The chain-of-thought paper demonstrated exactly this: make the model write out the intermediate steps and its performance changes. chain-of-thought

The mechanism is in how it guesses the next token: its thinking is its output, and writing more is computing more. Once intermediate steps are written, they enter the sequence, and every later step can see the earlier reasoning — which amounts to a few extra chances to catch a mistake.

Three caveats, plus a variant.

One, it costs. Intermediate steps are output too — slower and more expensive. Simple tasks do not need it.

Two, reasoning models do not need the line. The internal step already happens; asking again can interfere.

Three, you can have it put reasoning and conclusion in separate places. Reasoning gathered on one side, conclusion on the other, so your code can take only the conclusion and never show the reasoning to users.

The variant: have it review its own work before delivering. Ask it to go back over its answer against each constraint listed above and correct anything that does not comply. This pays off especially well when the constraints are hard — word counts, banned words, required elements.

None of them are free switches

These four moves are easy to treat as universal switches. They are not.

The payoff depends on the task. Thinking before answering helps a lot on multi-step reasoning and does essentially nothing for "translate this sentence." Reference anchoring is strongest when a real reference exists; without one you are back to adjectives. Having it question you only helps when the requirements really are fuzzy — when they are clear, those extra rounds are pure waste. The test is whether you actually have the condition the technique treats.

The payoff also depends on model capability. The same "write your steps out first" may buy nothing but a verbose restatement on a weaker model, and genuinely better reasoning on a stronger one. So a technique's value changes when the model changes — after swapping models, re-run your samples rather than assuming the technique still works.

And one deeper point: none of these moves change the fact that it is not checking facts. They shape the output's form, tone, and reasoning path, not its accuracy. Accuracy comes from supplying material, from checkable evidence, and from the final verification step.

Which puts techniques last in the order of work: fill in the four parts and get each line into the right layer first, then consider adding moves. And add them one at a time, comparing against the same sample set to see whether the move did anything — how to iterate on a prompt covers the method.

That closes out the writing. One question matters more than any technique: whether this job should go to the model at all. It completes the implementation; it does not supply the judgment — a line you can draw in three seconds.