Building AI Agents for SAP: Architecture, Tool Use, and Practical Pitfalls

The standard pitch for AI in SAP migration is “analyse your code and suggest fixes.” What that actually requires is a controlled system: agents that can read SAP artefacts, reason about them, invoke tools, handle errors, and hand work to other agents — without inventing API names or bypassing authorisation checks. This post covers how to build that system correctly.

The Agentic Loop

Every LLM-based agent, regardless of framework, operates the same fundamental loop:

1. Perceive — receive context (task description, prior observations, tool results)
2. Plan — reason about the next action (chain-of-thought, structured output)
3. Act — call a tool or produce a final answer
4. Observe — receive the tool result, update state, loop

The critical insight for SAP workloads is that the Perceive step must be heavily curated. An agent receiving a raw 800-line ABAP program will either truncate it (losing context) or waste tokens on boilerplate. Effective SAP agents pre-process: they extract the object dependency graph (tables, FMs, classes referenced), verify those objects against the target system, and feed only the structured analysis to the LLM planner.

Tool-Use Patterns for SAP APIs

LLMs invoke SAP systems via tool calls (function calling in the model API). The tool interface hides SAP-specific complexity — authentication, RFC/OData protocol, error handling — behind a clean function signature the model can reason about.

A minimal SAP tool set for ABAP analysis:

// Tool: read_abap_source
// Fetches ABAP source code from the connected system via RFC_READ_TEXT or ADT REST API
{
  name: "read_abap_source",
  description: "Retrieve the source code of an ABAP object by type and name.",
  parameters: {
    object_type: { type: "string", enum: ["PROG", "CLAS", "FUNC", "INTF"] },
    object_name: { type: "string" }
  }
}

// Tool: check_object_release_status
// Queries TADIR + release metadata to confirm C1/C2/deprecated status
{
  name: "check_object_release_status",
  description: "Check if a repository object has C1 release status in the target S/4HANA system.",
  parameters: {
    object_name: { type: "string" },
    object_type: { type: "string" }
  }
}

// Tool: run_atc_check
// Triggers ATC check via ADT REST API and returns findings
{
  name: "run_atc_check",
  description: "Run the S/4HANA readiness ATC variant on a named ABAP object and return findings.",
  parameters: {
    object_name: { type: "string" },
    check_variant: { type: "string", default: "S4HANA_READINESS" }
  }
}

Tool descriptions matter enormously. The model reads them verbatim during planning. Vague descriptions produce incorrect tool calls. Precise descriptions — including what the tool does not do — dramatically reduce hallucinated invocations.

Connecting to SAP via BTP Destinations

Direct credential injection into agent prompts is never acceptable. The correct architecture uses SAP BTP Destination Service as the credential broker.

The agent runtime (running in Cloud Foundry or Kyma) holds a BTP service binding. When it needs to call an SAP system, it:

1. Fetches a destination by name from the Destination Service.
2. Receives the resolved URL and a short-lived OAuth token (Principal Propagation or technical user, depending on the destination configuration).
3. Calls the SAP system REST/RFC endpoint using that token.
4. Tokens are never stored — each agent invocation fetches a fresh one.

This approach supports on-premise S/4HANA via the Cloud Connector, which tunnels the RFC/OData calls without exposing the SAP system directly to the internet.

// BTP destination resolution — credentials never touch the agent prompt
async function getSAPClient(destinationName: string): Promise<SAPClient> {
  const destination = await fetchDestination(destinationName); // BTP Destination Service SDK
  return new SAPClient({
    baseUrl: destination.url,
    authHeader: await destination.getAuthHeader() // short-lived token
  });
}

Multi-Agent Orchestration: Assessment → Remediation → Test

A single agent context window cannot reliably handle a full ABAP program portfolio analysis. The practical architecture uses specialised subagents coordinated by an orchestrator.

sequenceDiagram
    participant User
    participant Orchestrator
    participant A1 as A1 Readiness Agent
    participant A2 as A2 Remediation Agent
    participant A3 as A3 Test Gen Agent
    participant SAP as SAP System (via BTP)

    User->>Orchestrator: "Assess package ZSD_CUSTOM"
    Orchestrator->>A1: Dispatch readiness scan for ZSD_CUSTOM
    A1->>SAP: read_abap_source(PROG, ZSD_BILLING)
    SAP-->>A1: Source code
    A1->>SAP: check_object_release_status(BSEG, TABL)
    SAP-->>A1: C2 / deprecated
    A1->>SAP: run_atc_check(ZSD_BILLING, S4HANA_READINESS)
    SAP-->>A1: 14 findings (3 high, 8 medium, 3 low)
    A1-->>Orchestrator: Readiness report + risk score 7.2/10

    Orchestrator->>A2: Remediate high-priority findings in ZSD_BILLING
    A2->>SAP: read_abap_source(PROG, ZSD_BILLING)
    SAP-->>A2: Source code
    A2->>SAP: check_object_release_status(I_GLAccountLineItem, DDLS)
    SAP-->>A2: C1 released
    A2-->>Orchestrator: Remediated source + diff

    Orchestrator->>A3: Generate ABAP Unit tests for remediated ZSD_BILLING
    A3->>SAP: read_abap_source(PROG, ZSD_BILLING) [remediated version]
    SAP-->>A3: Remediated source
    A3-->>Orchestrator: Test class ZSD_BILLING_TEST with 12 test methods

    Orchestrator-->>User: Assessment report + remediation diff + test suite

The orchestrator’s role is coordination and state management, not analysis. It decides which agents to dispatch, in what order, with what context, and how to aggregate their outputs. It does not attempt to do the analysis itself.

Architecture Diagram

flowchart TD
    subgraph User Layer
        UI[Web UI / API Client]
    end

    subgraph Agent Platform
        ORC[Orchestrator\nworkflow state, agent dispatch]
        A1[A1 Readiness\nABAP rule engine + LLM]
        A2[A2 Remediation\ncode rewrite + diff]
        A3[A3 Test Gen\nABAP Unit scaffolding]
        A4[A4 Docs Gen\nspec reverse-engineering]
    end

    subgraph SAP Layer
        BTP[BTP Destination Service\ncredential broker]
        CC[Cloud Connector\non-premise tunnel]
        S4[S/4HANA System\nADT REST + RFC]
    end

    subgraph Data Layer
        DB[(Supabase\nscan results, reports)]
        GH[GitHub\nabapGit serialized objects]
    end

    UI --> ORC
    ORC --> A1 & A2 & A3 & A4
    A1 & A2 & A3 & A4 --> BTP
    BTP --> CC --> S4
    ORC --> DB
    A2 --> GH

Practical Pitfalls

1. Hallucinated ABAP Object Names

LLMs will generate plausible-looking ABAP identifiers that do not exist. A model might suggest I_MaterialDocument when the correct view is I_MaterialMovement. The mitigation is mandatory object verification before any suggested object name is used in code generation:

• Every object name produced by A2 Remediation must pass through the check_object_release_status tool before being written into output code.
• If the tool returns “not found,” the agent must backtrack and search for the correct alternative — not proceed with the unverified name.

2. Missing Authorization Checks

Agents generating ABAP code will frequently omit AUTHORITY-CHECK statements. The original ECC code may have them; the LLM does not reliably carry them through. A post-generation validation step must scan output code for:

• Object-level access (database reads/writes) without a preceding authority check
• RAP behavior implementations missing check_before_save
• OData service bindings without @MBC authorization annotations

Build this as a deterministic rule check, not another LLM call. Regular expression and AST-based analysis is more reliable than asking the model to self-review.

3. Context Overflow on Large Programs

A 3,000-line ABAP program with deeply nested includes will exhaust context windows or produce degraded analysis. Effective chunking strategies:

• Analyse at the include level for very large programs.
• Pass only the relevant procedure (function module, method) plus its extracted dependencies to the model.
• Use the orchestrator to maintain a cross-chunk shared state (object dependency graph, finding accumulator).

4. Non-Deterministic Outputs in CI

Agent outputs feeding a CI/CD pipeline must be deterministic enough to be reviewable. Use low temperature (0.1–0.2) for code generation tasks, structured output modes (JSON schemas) for findings, and mandatory human review gates before any generated code enters a transport request.

5. Rate Limits and Latency at Scale

A portfolio of 10,000 ABAP objects processed concurrently will hit both SAP system RFC load limits and LLM API rate limits. Design the agent dispatch layer with:

• A work queue (BTP Job Scheduling or a simple Supabase queue table) rather than direct parallel invocation.
• Configurable concurrency limits per SAP system connection.
• Idempotent task execution — completed objects should not be re-analysed on resume.

What Makes an Agent Actually Useful

The differentiator between a toy demo and a production SAP AI agent is the verification layer. Every fact the agent acts on — object existence, release status, field names, transport configuration — must be confirmed against the live system, not generated from training data. SAP object names, field names, and release status all change across ECC releases and S/4HANA versions. An agent that caches assumptions will produce wrong output silently.

Build the verification step as a first-class tool, run it before every code generation action, and treat an unverifiable object as a hard stop requiring human input. The extra round trips are worth it.