The SAP AI Stack: AI Foundation, Generative AI Hub & Joule
Lesson 8 — Five SAP AI product names, five completely different jobs, and one confusion that quietly costs projects real budget: buying or building the wrong layer because two names sound alike. Here's the map that ends that.
Prerequisites
- Lesson 7: From chatbot to agent
Here’s a conversation that happens more often than SAP would probably like: a project team spends a sprint evaluating “the Generative AI Hub” for a use case that actually needed Joule Studio, or scopes out “AI Foundation” work that was really an AI Agent Hub governance question. Nobody was incompetent. The names are just genuinely confusing, and confusing names cost real budget when they drive the wrong architecture decision.
You understand LLMs, prompting, RAG, and agents in the abstract now. Time to map every one of those words onto SAP’s actual platform — and the biggest source of confusion here isn’t the technology at all. It’s the names.
Six layers, six distinct jobs — don’t let any two blur together
Think of SAP’s AI as a stack, and hold each layer to a strict, separate job description:
- SAP BTP — the cloud platform everything else in this list runs on top of.
- AI Foundation — the AI services layer on BTP: tools, foundation models, grounding, lifecycle services. The platform for building AI, full stop.
- Generative AI Hub — inside AI Foundation, gives you access to many LLMs, both SAP-hosted and hyperscaler, through one harmonised API, plus an orchestration service. It is not “SAP’s one LLM.” It’s a model-access and orchestration layer, and that distinction matters the moment you’re scoping a project.
- SAP HANA Cloud Vector Engine — where embeddings live (Lesson 4), enabling RAG natively inside the database you already run.
- Joule — the assistant your end-users actually talk to.
- Joule Studio — the low-code tool where you build custom agents (Lesson 11), distinct from Joule itself.
- AI Agent Hub — the registry and governance layer for managing agents once you have more than one (Lesson 13).
If you take one sentence out of this entire lesson, make it this: AI Foundation is not the Generative AI Hub, which is not Joule, which is not Joule Studio, which is not the AI Agent Hub. Mixing any two of these up is the single most common way project teams end up scoping the wrong piece of work.
The orchestration service is where every earlier lesson actually lives
Everything you learned in Lessons 2 through 6 is delivered, in production, by the Generative AI Hub’s orchestration service — in one API call, it can handle grounding (Lesson 5), prompt templating (Lesson 3), data masking (anonymising or pseudonymising PII before a prompt ever reaches a model), and content filtering (blocking unsafe input or output).
This is also why SAP can swap the underlying model without you rewriting a line of application code: orchestration and grounding stay under SAP’s control, while the model itself is a genuinely swappable component. You’ll see this exact same decoupling again, applied specifically inside Joule, in the very next lesson.
The part nobody wants to say out loud: AI Units
Generative AI consumption is metered, and it’s worth being precise rather than vague here, because vague is where expensive mistakes hide. Technical metering measures usage from the underlying GenAI tokens — the exact economics you sized in Lesson 2, now made real. Commercial billing happens through AI Units — a finite, consumable entitlement.
The rule that matters, and one of this entire project’s non-negotiable principles: AI Unit pricing is not published, and it’s negotiated commercially. Size your token usage first, using the skill from Lesson 2. Have the pricing conversation second, directly with SAP. Anyone who quotes you a firm public AI-Unit price without checking current terms is guessing — and repeating that guess to a client is how consultants lose credibility over something that was entirely avoidable.
Where this shows up — the full reference map
For a fuller, visual tour of these layers — including the Joule / Joule Studio / SAP-ABAP-1 / MCP distinctions in more depth — see our companion guide: A Plain-English Map of SAP’s AI Stack. This lesson is the foundations-series version; that post is the reference card you keep open in a second tab during a scoping call.
Try it yourself
If you have a BTP entitlement with Generative AI Hub access — a trial or free tier may be available, but verify current availability rather than assuming it:
- Open the Generative AI Hub and send one orchestration request combining a prompt template with a grounding or data-masking step.
- As it runs, label each part against what you already know: which piece is prompting (Lesson 3)? Which is grounding (Lesson 5)? Which is masking, new to this lesson?
No entitlement handy? Follow any documented Generative AI Hub walkthrough and do the same labelling exercise on paper. The point isn’t memorising a UI that will change by next quarter — it’s learning to recognise your own fundamentals hiding inside SAP’s managed tooling.
Key takeaways
- SAP AI is a layered stack: BTP → AI Foundation → Generative AI Hub (model access + orchestration) → Joule → Joule Studio → AI Agent Hub.
- The Generative AI Hub is multi-model access plus an orchestration service — not a single SAP-built LLM.
- Vectors live in the HANA Cloud Vector Engine; the model stays swappable while grounding stays under SAP’s control.
- Consumption meters from GenAI tokens; billing runs through AI Units — finite, negotiable, and never a number you should quote without checking first.
Next: zoom all the way into the assistant layer — exactly how Joule turns a request into a grounded, governed action, step by step.