From Chatbot to Agent: Tools, Function Calling & the Reason–Act Loop

Here’s a sentence almost everyone gets backwards the first time they hear it, and getting it wrong changes how safely you’d design a production agent: does the AI model actually run the code, or does it just ask for something to be run? Get the answer wrong, and you’ll design authorization and approval controls in the wrong place entirely.

The model can now know, through RAG, and answer, through prompting. An agent can do — check an order’s status, create a record, kick off a workflow. This lesson is the conceptual key that unlocks everything SAP-specific from here on, so read it carefully; it’s short, but it’s load-bearing.

The detail almost everyone gets wrong

Say this one twice, because it’s the whole lesson: the model does not execute the tool. It only emits a structured request — “call get_order_status with order_id = 123.” Your runtime — the code you wrote and control, sitting around the model — actually executes that function, and hands the result back.

The LLM is the decider. Your code is the doer. That separation is not a technical footnote; it is exactly where you place safety, authorization, and approval — because you control the runtime completely, and the model’s whims never get to touch anything directly.

A tool is a name, a description, and a promise

You expose a tool to the model as a definition: a name, a description, and a parameter schema, typically JSON Schema. For example:

{
  "name": "get_sales_order_status",
  "description": "Returns the current status of a sales order by its ID.",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": { "type": "string", "description": "The 10-digit sales order number" }
    },
    "required": ["order_id"]
  }
}

Two things matter here more than the JSON syntax itself. The description is prompt engineering in disguise — the model chooses which tool to call based on that description, so a vague one produces a wrong tool call, silently, with no error message telling you why. And structured output is what makes any of this possible — the same “make the output machine-shaped” skill from Lesson 3, now doing genuinely load-bearing work instead of a nice-to-have.

The reason–act loop

This loop is inherently non-deterministic — the model is deciding, step by step, what to do next. Which is exactly why you wrap it in deterministic guardrails: a maximum number of steps, an explicit allow-list of which tools even exist, and human approval before anything consequential happens. The loop can improvise. The rails around it should never have to.

Where this shows up in SAP

In SAP, a skill is a single discrete capability, and an agent is a goal-driven orchestration of several skills or tools — precisely this loop, managed for you inside Joule Studio (Lesson 11). The tools an agent calls are exposed through open protocols, chiefly MCP (Lesson 10), and the runtime that actually executes them is SAP’s — which is exactly where authorization and governance live (Lesson 13), because that’s where the doing happens, not where the deciding happens.

Try it yourself

1. Write a JSON tool definition for get_open_invoices(customer_id, days_overdue) — name, description, parameter schema, required fields, the works.
2. Hand-trace one full turn: a user asks “which invoices for customer 1000001 are more than 30 days overdue?” What call does the model emit? What does your runtime actually return? What does the model do with that result next?

You’ve just designed the contract between an LLM and your systems. That contract — not the model itself — is the thing you’re going to spend the rest of this series learning to secure.

Key takeaways

• An agent is an LLM plus tools plus a loop that reasons, acts, observes, and decides — repeatedly, until the goal is met.
• The model emits a tool call; your runtime executes it. Confusing these two is the single most common design mistake in agent architecture.
• A tool is a name, a description, and a JSON parameter schema — and the description quietly drives which tool gets picked.
• Structured output is what makes tool calling possible at all.
• Wrap the loop in deterministic guardrails: step limits, allow-lists, and human approval before anything consequential.
• In SAP: skills are discrete capabilities, agents orchestrate them, and Joule Studio is where you build one.

That completes the vendor-neutral fundamentals. From here on, every lesson maps this exact vocabulary onto SAP’s real platform — starting with the stack itself.