BTP & Integrationintermediate

Generative AI Hub Part 2: Deployments, Prompts and Consuming Models

Key takeaway

Get hands-on: create an orchestration deployment, call it with the SAP Cloud SDK for AI (Python, Java or JavaScript), understand prompt templating, model configuration, and how a single orchestration deployment lets you swap models without redeploying — plus restricting which models are allowed.

Explanation

📌 KEY POINTS
✅ A single orchestration deployment configures and consumes models — no need to deploy each model individually.
✅ The orchestration pipeline has mandatory modules (templating, model configuration) and optional ones (grounding, data masking, content filtering) shown/hidden via Edit.
✅ The SAP Cloud SDK for AI (Python/Java/JavaScript) is the recommended way to call the orchestration service from custom code.
✅ You can explicitly restrict which LLMs are usable in a deployment — important for compliance and cost control.
⚠️ Skipping model restriction lets any allowed model be swapped in later, which can silently change behavior, cost or compliance posture.

━━━━━━━━━━━━━━━━━━━━
▶ WHERE WE ARE
Part 1 explained the relationship. Now deployments and calls.

━━━━━━━━━━━━━━━━━━━━
▶ 1. CREATING A DEPLOYMENT

Resource group → configuration → deploy → wait for Running status.

━━━━━━━━━━━━━━━━━━━━
▶ 2. PIPELINE MODULES

Templating and model configuration (mandatory); grounding, masking, filtering (optional).

━━━━━━━━━━━━━━━━━━━━
▶ 3. CALLING IT

SAP Cloud SDK for AI; REST via AI_API_URL.

━━━━━━━━━━━━━━━━━━━━
▶ 4. MODEL RESTRICTION

Approved model list per deployment.

━━━━━━━━━━━━━━━━━━━━
▶ 5. TESTING

AI Launchpad playground before code integration.

━━━━━━━━━━━━━━━━━━━━
▶ STEP BY STEP

Create an orchestration deployment in a resource group.
Wait for Running status.
Call it with the SDK using a simple prompt template.

✅ CHECKPOINT: a real response comes back through your own code, not just the playground.

Code example

ABAP Code

===== DEPLOYMENT FLOW =====Resource group → Configurations → Deployment Details → Create Deployment → status: Running ===== SDK CALL (conceptual, Python) =====from gen_ai_hub.orchestration.service import OrchestrationServiceconfig = OrchestrationConfig(template=my_template, llm=my_llm_config)service = OrchestrationService(api_url=deployment_url, config=config)response = service.run(template_values={...}) ===== MODEL RESTRICTION =====Deployment config → allowed models list → enforce internal standard / cost control

Real project scenario

A developer called the orchestration deployment URL immediately after clicking Create Deployment and got errors. Waiting for the status page to show Running resolved it — a five-minute wait, not a bug.

Common mistakes

Calling before status is Running.
Iterating prompts only in code (slow feedback loop).
Not restricting models, allowing silent swaps.
Hard-coding the deployment URL instead of reading it from configuration.
Skipping the SDK in favor of raw HTTP without understanding auth.

Best practices

Wait for Running status.
Prototype in the playground first.
Restrict models explicitly.
Store deployment URLs in configuration, not code.
Use the official SDK for supported languages.

Interview angle

Questions: "How do you create an orchestration deployment?", "Why use one deployment instead of many model deployments?", "How do you restrict model usage?"
Strong answer: deployment creation and status, pipeline modules, SDK usage, model restriction, playground-first testing.

Practice this lesson

Quick check

1. What does a single orchestration deployment allow you to do?

2. Which SAP proprietary model assists with ABAP code explanation and generation?

3. A developer calls a deployment URL immediately after clicking Create Deployment and gets errors. What is the likely cause?

Interview questions

1. A project team is working on Generative AI Hub and SAP AI Core. Using the Production GenAI on SAP: Grounding, Orchestration, Evaluation and Guardrails scenario, how would you approach the requirement from design through validation?

Answer it in your own words first, then compare.

2. What design decisions and trade-offs matter most when implementing Generative AI Hub and SAP AI Core, particularly for Generative AI Hub and SAP AI Core Fundamentals?