BTP & Integrationadvanced

Generative AI Hub Part 3: Production Patterns — Grounding, Data Masking, Content Filtering and Evaluation

Key takeaway

Move from prototype to production: grounding responses in your own data (retrieval-augmented generation), masking sensitive data before it reaches the model, content filtering for safety, inference observability, and evaluating outputs before trusting them in business processes.

Explanation

📌 KEY POINTS
✅ Grounding connects the model to your own business data (documents, tables) so answers are based on facts, not just training data — reduces hallucination.
✅ Data masking strips or pseudonymizes sensitive fields before they reach the model, and can restore them in the response.
✅ Content filtering blocks unsafe or policy-violating inputs/outputs at the pipeline level, not just in application code.
✅ Inference observability and evaluation let you measure quality, cost and latency over time — essential before an assistant touches real business decisions.
⚠️ An assistant that produces fluent, confident, but factually wrong answers is more dangerous than one that visibly fails — always evaluate against ground truth before production rollout.

━━━━━━━━━━━━━━━━━━━━
▶ WHERE WE ARE
Part 2 covered deployments and calls. Now production hardening.

━━━━━━━━━━━━━━━━━━━━
▶ 1. GROUNDING

RAG pattern; connecting business data.

━━━━━━━━━━━━━━━━━━━━
▶ 2. DATA MASKING

Sensitive fields in/out of the pipeline.

━━━━━━━━━━━━━━━━━━━━
▶ 3. CONTENT FILTERING

Safety at the orchestration layer.

━━━━━━━━━━━━━━━━━━━━
▶ 4. OBSERVABILITY

Logging, cost, latency tracking.

━━━━━━━━━━━━━━━━━━━━
▶ 5. EVALUATION

Ground truth comparison before rollout.

━━━━━━━━━━━━━━━━━━━━
▶ STEP BY STEP

Enable a grounding module pointing to your document set.
Add data masking for personal data fields.
Run an evaluation comparing outputs to known correct answers.

✅ CHECKPOINT: you can show accuracy, cost and latency numbers, not just "it feels good."

Code example

ABAP Code

===== PIPELINE WITH PRODUCTION MODULES =====Templating → Data Masking (in) → Model Configuration → Grounding (retrieval) → Content Filtering → Data Masking (restore) → Response ===== EVALUATION APPROACH =====Test set of Q&A pairs → run through deployment → compare model answer vs expected → score accuracy → track over model/version changes ===== OBSERVABILITY =====Log: prompt template used, tokens consumed, latency, model version, filter triggers

Real project scenario

An internal assistant was allowed to answer HR questions using an ungrounded general model; it gave confident but outdated policy answers. Adding a grounding module pointing to the current HR policy documents, plus an evaluation set of known Q&A pairs, caught the regressions before rollout.

Common mistakes

Shipping without grounding for factual domains.
Trusting fluent output as correct output.
Skipping evaluation against ground truth.
Not verifying masking end-to-end.
No observability until something breaks.

Best practices

Ground answers in real business data.
Mask sensitive fields verifiably.
Filter content at the pipeline layer.
Evaluate against known-correct answers before go-live.
Track cost, latency and accuracy continuously.

Interview angle

Questions: "What is grounding and why does it matter?", "How do you handle sensitive data in prompts?", "How do you evaluate an LLM-based assistant before production?"
Strong answer: grounding/RAG, masking, content filtering, observability, evaluation discipline.

Practice this lesson

Quick check

1. An internal assistant gave confident but outdated policy answers because it wasn't connected to current documents. What should be added?

2. During hypercare, what should be tracked continuously for a generative AI hub deployment?

3. An AI reviewer produced accurate normal-case answers but confidently wrong answers on edge cases that were never tested. What was missing?

Interview questions

1. You inherit an existing Generative AI Hub and SAP AI Core implementation covering Production GenAI on SAP: Grounding, Orchestration, Evaluation and Guardrails. What would you review before changing it, and what would make you confident the change is production-ready?

Answer it in your own words first, then compare.

2. A project team is working on Generative AI Hub and SAP AI Core. Using the Production GenAI on SAP: Grounding, Orchestration, Evaluation and Guardrails scenario, how would you approach the requirement from design through validation?