System Prompt vs User Prompt: When to Use Each

Most people treat the system prompt as a place to dump a giant wall of instructions, then wonder why the model ignores half of them. Others put everything in the user turn and get inconsistent behavior across conversations. The real issue is simpler: the two slots are not interchangeable, and where you place an instruction changes how strongly the model weights it. This post gives you a clear mental model for splitting instructions correctly — with a decision table you can apply immediately.

Why the split matters

Chat-based models — GPT-4o, Claude 3.x, Gemini 1.5, and their successors — are trained with a three-part message structure: system, user, and assistant. The system message is processed before the conversation begins. It sets the model's operating context: who it is, what it's allowed to do, and what shape its responses should take. The user message is treated as runtime input — the thing the human actually says in the moment.

This distinction is not cosmetic. Models are fine-tuned to treat system-level instructions as persistent rules and user-level instructions as requests. Put a persistent rule in the user turn and you're asking the model to treat a request as a rule — it often does, but less reliably, especially in long conversations where earlier turns get compressed or dropped.

The practical consequence: instructions that need to survive the whole conversation belong in the system prompt; instructions that are specific to one exchange belong in the user prompt.

The instruction hierarchy in practice

Think of it as two layers of authority:

Layer	Slot	Typical weight	Survives context rollover?
Standing orders	System prompt	High	Yes
Per-turn requests	User prompt	Medium	Only while in context
Model's priors	Training	Baseline	Always

"Standing orders" are things like tone, persona, output format, forbidden topics, and language. "Per-turn requests" are the actual task: summarise this document, translate this paragraph, write a subject line for this email.

When you put a standing order in the user turn, the model treats it as a request that might be overridden by a later request. That's why "always respond in formal English" in the user message sometimes gets quietly abandoned three exchanges in — the model sees a new user turn and re-evaluates.

What belongs in the system prompt

Use the system prompt for anything that should be true of every response in the session, regardless of what the user asks:

• Persona and role — "You are a concise legal research assistant. You do not give legal advice."
• Output format defaults — "Respond in plain text. No markdown unless the user explicitly requests it."
• Scope limits — "Only answer questions about the company's products. Politely decline anything outside that scope."
• Language and tone — "Write at a 9th-grade reading level. Avoid jargon."
• Safety rails — "Never reproduce copyrighted text verbatim."
• Tool or API context — "The current date is {{date}}. The user's account tier is {{tier}}."

A useful test: if this instruction were missing from every response, would the output be wrong? If yes, it belongs in the system prompt.

What belongs in the user prompt

Use the user prompt for anything specific to the task at hand:

• The actual content to process ("Here is the contract. Summarise the termination clauses.")
• Task-specific constraints that only apply this turn ("Keep this underundefinedwords — it's for a mobile push notification.")
• Dynamic variables the system prompt can't know in advance ("The audience for this email is mid-level finance managers in Australia.")
• Clarifications that refine a standing order for one response ("This time, use a bulleted list instead of prose.")

The user prompt is where the work lives. Keep it focused on the job, not on re-establishing who the model is.

The grey zone: format instructions

Format is the most commonly misplaced instruction type. Many people put format rules in the user prompt because they feel like part of the request. But if you want consistent formatting across an entire session — say, all outputs as JSON, or all responses starting with a one-sentence summary — that's a standing order. Put it in the system prompt.

If you only need a specific format for one output, put it in the user prompt, and be explicit: "Return this as a markdown table with three columns: Term, Definition, Example." One-off format instructions in the user turn work fine; they just don't persist.

A worked example: customer support bot

Weak version (everything in the user prompt):

You are a helpful support agent for Norvik Software. Always be polite and concise. Don't discuss competitors. Now help me: I can't log in to my account.

This works for the first message. But by message five, the model may have drifted — it's no longer certain whether "don't discuss competitors" is still in effect, because that instruction is buried in a now-distant turn.

Stronger version (split correctly):

System prompt:

You are a support agent for Norvik Software. Tone: professional and concise. Never mention competitor products by name. If a user asks about billing, direct them to support@norvik.com. Respond in plain text only.

User prompt (turn 1):

I can't log in to my account.

Now the standing orders are locked in. The user turn contains only the actual request. The model doesn't have to re-derive the rules from context — they're always present.

You can test how different placements affect model behavior using PromptCueLab's Refiner, which lets you edit both the system and user layers side by side and compare outputs.

Checklist: where does this instruction go?

Before you write a prompt, run each instruction through this filter:

• [ ] Should this apply to every response in the session? → System prompt
• [ ] Is this specific to the current task or content? → User prompt
• [ ] Does this involve who the model is (persona, role, constraints)? → System prompt
• [ ] Does this involve what the model should do right now? → User prompt
• [ ] Is this a format rule that must persist? → System prompt
• [ ] Is this a one-off format request? → User prompt
• [ ] Does this include dynamic content (user-supplied text, documents, variables)? → User prompt

Key takeaways

• System and user prompts are not interchangeable — models weight them differently by design.
• Standing orders (persona, tone, scope, persistent format rules) belong in the system prompt where they survive the full conversation.
• Per-turn work (the actual task, dynamic content, one-off constraints) belongs in the user prompt.
• Mixing the two layers causes drift: instructions placed in the wrong slot get treated with the wrong level of authority.
• Format instructions are the most commonly misplaced type — decide whether the rule is persistent or one-off before choosing a slot.
• If you're building a multi-turn product or agent, a clean system/user split is not optional — it's the foundation.

If you're not sure which models handle system-prompt instructions most faithfully, browse the model library on PromptCueLab to compare how different chat models handle instruction hierarchy. And if you're building agentic workflows where system prompts are injected programmatically, the MCP integration lets you manage prompt layers directly from your AI client.

For teams that also need to match their prompt architecture to the right tooling stack, CraftMyStack is worth a look — it helps you evaluate which development tools and frameworks best support structured prompt management at scale.