Prompting That Works (+ Context Engineering)

Two consultants, same model, same client, same afternoon. One types “explain this ABAP” and pastes a subroutine, gets three paragraphs of vague prose, and spends the next twenty minutes re-asking. The other writes four extra lines of setup around the exact same code, and gets back exactly the structured answer they needed, first try. Nobody in this story has a better model. One of them just knows how to prompt.

That’s the whole pitch for this lesson. Prompting is the highest-leverage skill in this entire series — more leverage than the model you pick, more leverage than the platform you’re on — because it’s free, it’s immediate, and it compounds into everything downstream: retrieval, agents, all of it work better once you can reliably steer a probabilistic system toward the answer you actually need.

What separates a weak prompt from a strong one

Weak prompts fail for the same handful of reasons, over and over: no role, no clearly-bounded context, no single task, no specified output shape. A strong prompt supplies all four, every time:

• Role — a system message setting the persona and the rules. “You are an SAP ABAP reviewer. Only use facts from the code provided.”
• Context — the material to work on, clearly fenced with delimiters (triple backticks, XML-style tags) so the model can’t confuse your data with your instructions.
• Task — exactly one clear instruction. Not three vague ones stacked together.
• Output format — tell it precisely what shape you want back. “Return a list: purpose, inputs, side effects, risks.”

One honest caveat, because overclaiming here would undercut the whole point of this series: many APIs let you separate a system message from a user message, and the common belief is that system instructions always win. In practice, how strictly that holds is model-dependent — treat it as strong guidance, never as an ironclad guarantee you can build safety-critical logic on top of.

Zero-shot, few-shot, and knowing when to bother

Zero-shot means you just ask — fine for simple, common tasks. Few-shot means you show one to three worked examples of input-to-output before asking for real, which pins down format and tone far better than describing them in words ever will.

Few-shot is not automatically better, though. It costs more tokens (you did Lesson 2 — you know exactly what that means now), and it can over-constrain a task that didn’t need constraining. Reach for it specifically when output shape matters, or when zero-shot keeps coming back inconsistent.

Make the output something code can actually use

If anything downstream is going to read the answer — a script, an integration, an agent — stop asking for prose. Ask for a fixed structure: a JSON object, a defined list, something with a schema. Constraining the output shape is precisely what turns an LLM from a chat toy into a component you can build a system around. This becomes non-negotiable the moment we build agents in Lesson 7.

Context engineering: spending the desk wisely

Remember the desk-you-rent-by-the-token from Lesson 2? Context engineering is the discipline of deciding what actually earns a seat on it. Put the instructions and the genuinely relevant data in the prompt. Resist the temptation to paste “everything, just in case” — it costs tokens and, worse, it dilutes the model’s attention on the part that actually matters.

Hold this mental model and you’ll never mix these three up again: prompt is the instructions, context is the specific data for this exact task, and retrieval is how you get the right context in without paying to bloat the window with the wrong context — which is precisely Lesson 5.

Better prompting reduces wrong answers. It does not eliminate hallucination. No amount of clever phrasing makes a model reliably factual about data it was never given in the first place — and that hard limit is exactly why the next block of lessons exists.

Try it yourself

Take a genuinely weak prompt and rebuild it, side by side:

• Weak: “explain this ABAP” + pasted code, no structure at all.
• Strong:

  You are a senior ABAP reviewer. Analyse only the code between the tags. <code> …one subroutine, pasted… </code> Return exactly: 1) Purpose, 2) Inputs, 3) Side effects, 4) Risks. Be concise. If something isn’t determinable from the code, say “not determinable.”

Run both against the same model. The strong version comes back more consistent, more usable, and — because it explicitly forbids guessing — noticeably less likely to invent something that isn’t there.

Where this shows up in SAP

Inside Joule and the Generative AI Hub, SAP is already wrapping your request with system instructions, business context, and templating before it ever reaches a model — managed prompting you never have to hand-write. The moment you build a custom agent (Lesson 11), prompt and tool-description quality stop being SAP’s job and become yours, and everything in this lesson is what decides whether that agent behaves.

Key takeaways

• A strong prompt is role + delimited context + one task + an explicit output format — every time, no exceptions.
• System messages set standing rules; user messages carry the request — precedence between them is model-dependent, so don’t build safety on the assumption it’s absolute.
• Few-shot examples pin down format and tone; reach for them when shape matters, not as a default.
• Ask for structured output the moment anything downstream will read the answer programmatically.
• Context engineering means spending the window on what’s relevant and retrieving the rest — never dumping everything in “just in case.”
• Prompting reduces hallucination. It cannot eliminate it.

Next: how do you give a model knowledge it was never trained on in the first place? It starts, perhaps unexpectedly, with turning text into numbers.