How SAP Joule Works: Grounding, Orchestration & Agents

“Joule just knows things about our system” is the sentence that ends most people’s understanding of Joule, and it’s the sentence that stops them from ever being able to design anything with it. Joule isn’t magic. It’s the exact pieces you’ve already learned in this series, assembled in a specific order — and once you can name that order, you stop being a Joule user and start being someone who can actually architect around it.

The request lifecycle, in four steps

When a user asks Joule something, roughly this happens, every time: Joule understands intent — interpreting what’s actually being asked, aware of which SAP app the user is in and what their organisation has licensed. It grounds the request — enriching it with relevant business context before any model call happens at all; this is the RAG you learned in Lesson 5, wearing enterprise clothing. It orchestrates — the grounded, templated, masked prompt travels through the Generative AI Hub’s orchestration service to a model (Lesson 8). It acts — running the right skill or agent to fetch data or perform the actual transaction, then returning the result.

Here’s the sentence worth underlining: the value was never the model. It’s steps two and four — a swappable model, grounded in your governed data, acting strictly through your authorisations.

Two kinds of grounding, and conflating them is the mistake to avoid

Joule does not ground every request the same way, and this is the exact spot where explanations get sloppy. Unstructured content — documents, help text, policies — is grounded through RAGe, Retrieval-Augmented Generation for enterprise, which is precisely the embeddings-retrieve-augment pipeline from Lessons 4 and 5. Structured business data — orders, customers, materials, and the relationships between them — is grounded through the SAP Knowledge Graph instead.

Presenting RAGe as the whole story is the single most common mistake in explaining Joule, and now you know better than to make it. RAGe handles unstructured content; the Knowledge Graph handles structured data. Together, they let Joule answer both “what does our returns policy say?” and “which open orders does customer 1000001 have?” — two completely different questions, answered by two completely different mechanisms, seamlessly, from the user’s point of view.

Skills and agents are not the same word wearing different hats

A skill is one discrete, deterministic capability — call an API action, answer one kind of question, one job. An agent plans and orchestrates several skills or tools toward a multi-step goal — the exact reason-act loop from Lesson 7, now running on SAP’s infrastructure with metering, tracing, and security built in.

Get the timeline right, because dates matter here more than usual: SAP shipped the Skill Builder first; the Agent Builder reached general availability in January 2026. Which means “build your own agent” is a genuinely real, generally-available capability as of 2026 — but always date claims like this one out loud when you repeat them, because this tooling moves fast enough that a confident, undated claim ages badly within a quarter.

The privacy conversation, told straight

Per SAP’s own documentation: your data stays inside your tenant, Joule itself stores no customer data, sensitive fields can be masked before a prompt ever leaves, and partner LLMs are contractually barred from training on your data. Here’s the nuance worth being honest about rather than glossing over: “no training” is a contractual guarantee, not a purely technical one — masked content still physically reaches the provider’s infrastructure for inference; it simply isn’t retained or trained on afterward. For most enterprises, that’s a genuinely well-governed trade-off. It’s just worth knowing precisely, rather than assuming “nothing ever leaves our tenant” in the absolute sense, because a client’s security team will eventually ask you the precise version of this question.

Go deeper

This lesson is the series-level summary. For the full architecture — scenario catalog, knowledge catalog, the complete orchestration pipeline, and a full data-privacy breakdown — read the companion piece: How SAP Joule Works: A Technical Deep Dive.

Try it yourself

In a system with Joule enabled — availability is edition- and region-gated, so verify before you promise it in a proposal:

1. Run three different query types and classify each: find a record (structured → Knowledge Graph), explain a policy or field from a document (unstructured → RAGe), trigger a navigation or action (a skill).
2. Notice that every one of them respects your SAP authorisations — Joule genuinely cannot show a user data that user isn’t allowed to see, even if the underlying model, in isolation, technically “knows” the answer.

No access to a live system? Storyboard the same three interactions on paper, and label the grounding type and the authorisation each one would inherit.

Key takeaways

• Joule’s lifecycle is understand, ground, orchestrate, act. The value sits in grounding and governed action, never in the model alone.
• Two distinct grounding paths: RAGe for unstructured content, the SAP Knowledge Graph for structured data — never present RAGe as the whole story.
• Skill means one discrete capability; agent means multi-step orchestration. The Agent Builder went GA in January 2026 — date this claim whenever you repeat it.
• Data stays in your tenant; masking plus contractual no-train protect it — and it’s worth knowing the contractual-versus-technical distinction precisely, because a security review will eventually ask.

Next: the open protocols — MCP and A2A — that let agents reach tools, and reach each other.