Using Start, End, and Expert Routines for Custom Transformation Logic

Explanation

Standard field-level rule types in a BW transformation (direct assignment, constant, formula, read master data) cover most mapping needs, but many real transformations require logic that spans multiple records in the transfer package, needs external lookups against master data or other DataStore objects, or must aggregate/split records conditionally. This is where routines come in. A start routine executes once per data package before field-level rules run; it receives an internal table of source records (SOURCE_PACKAGE) that you can modify, delete rows from, or use to build lookup tables for later field routines. An end routine executes once per package after field-level mapping, operating on the result package (RESULT_PACKAGE) structured like the target; this is the natural place to consolidate duplicate keys, apply final business rules, or delete records that fail late-stage validation. An expert routine replaces the entire transformation for a rule group, giving full control over both source and result packages, but it forfeits the maintainability of individual field rules, so it is reserved for genuinely complex, non-decomposable logic. Design discipline matters here. Start routines are the right place to do package-wide operations once, such as selecting master data texts or exchange rates for the whole package into an internal table, rather than issuing a database read for every single record in a field routine (which is a classic performance anti-pattern). A well-written start routine builds a sorted internal table or hashed table keyed appropriately, then field routines or the end routine look up values from that table in memory instead of hitting the database repeatedly. This single practice is often the difference between a load that finishes in minutes and one that times out. End routines are commonly used to eliminate duplicate keys before they are written to a DataStore object with overwrite semantics, to net values across split source lines, or to apply currency translation types that depend on the fully mapped record. Because the end routine works on RESULT_PACKAGE, you can also use it to raise transformation-level errors and route bad records to the error stack, provided the DTP is configured to support error handling for that transformation step. Expert routines exist for edge cases: source-to-target cardinality that changes dynamically, complex looping logic that cannot be expressed as a chain of start/field/end routines, or migration scenarios copying legacy custom ABAP logic wholesale. They should be used sparingly because they are opaque to non-ABAP-literate team members, harder to test in isolation, and bypass the transformation's visual rule documentation. In BW/4HANA, the same routine concepts apply, but SAP recommends pushing logic to the database (AMDP-based routines, i.e., ABAP Managed Database Procedures) where volumes are large, since AMDP routines execute inside HANA and avoid ABAP-to-database round trips. Classic ABAP routines still work but may be slower on very large packages. In S/4HANA embedded analytics scenarios using CDS-based virtual data models, transformation logic is often replaced entirely by CDS view calculations, so BW-style routines apply mainly to classic or hybrid BW-on-S/4HANA setups, not to native embedded analytics queries. Testing routines requires more than checking a single record: you must test package boundary behavior (what happens to a lookup built in a start routine when a package split changes group membership) and verify that routines behave correctly when a package contains zero records, duplicate keys, or unexpected nulls, since these edge cases are common causes of production data quality incidents.

Code example

* Example: START ROUTINE - build a lookup table once per packageMETHOD start_routine.  DATA: lt_material TYPE STANDARD TABLE OF /bi0/pmaterial,        ls_source   LIKE LINE OF SOURCE_PACKAGE.   " Collect distinct material numbers from the whole package  DATA: lt_matnr TYPE STANDARD TABLE OF /bi0/oimaterial.  LOOP AT SOURCE_PACKAGE INTO ls_source.    APPEND ls_source-material TO lt_matnr.  ENDLOOP.  SORT lt_matnr. DELETE ADJACENT DUPLICATES FROM lt_matnr.   " Single read for the whole package instead of per-record reads  IF lt_matnr IS NOT INITIAL.    SELECT * FROM /bi0/pmaterial      INTO TABLE lt_material      FOR ALL ENTRIES IN lt_matnr      WHERE material = lt_matnr-table_line.  ENDIF.   " Store lt_material in a global class attribute for reuse in field routines  gt_material_lookup = lt_material.ENDMETHOD. * Example: END ROUTINE - remove duplicate keys before load to overwrite DSOMETHOD end_routine.  DATA: ls_result LIKE LINE OF RESULT_PACKAGE.  SORT RESULT_PACKAGE BY customer material fiscper.  DELETE ADJACENT DUPLICATES FROM RESULT_PACKAGE    COMPARING customer material fiscper.ENDMETHOD.

Real project scenario

A retail customer's sales DataStore object load was timing out during month-end because a field routine performed a database SELECT SINGLE against a master data attribute table for every one of several hundred thousand records in each package. The consulting team refactored the logic: a start routine now performs one bulk SELECT FOR ALL ENTRIES for all distinct keys in the package, stores results in a sorted internal table on a global class attribute, and the field routine performs an in-memory READ TABLE with binary search instead of a database hit. Load time for the month-end package dropped from over 90 minutes to under 12 minutes, and the change was documented with before/after runtime statistics from process chain monitoring to justify the effort to the customer's steering committee.

Common mistakes

• Performing database SELECTs inside field-level routines that execute once per record instead of once per package in the start routine, causing severe performance degradation on large packages • Using an expert routine to solve a problem that could be cleanly expressed with standard rule types plus a start/end routine, sacrificing maintainability • Not initializing or resetting global class attributes between package executions, causing stale lookup data to leak across packages in parallel DTP processing • Deleting records in a start routine without also updating the monitor/error handling so operations teams cannot see why record counts dropped • Assuming end routine RESULT_PACKAGE changes automatically cascade to downstream key figure aggregation without explicitly handling summarization logic

Best practices

• Perform bulk data reads (master data, DSO lookups) once per package in the start routine, not per record in field routines • Reserve expert routines for logic that genuinely cannot be decomposed into start/field/end routines • Use end routines to deduplicate, consolidate, or apply record-spanning business rules after field mapping is complete • Document routine logic with inline comments referencing the business rule or ticket number driving the requirement • In BW/4HANA, evaluate AMDP-based routines for high-volume loads to push processing into the HANA database layer • Test routines against edge cases: empty packages, duplicate keys, and packages split by semantic groups

Interview angle

Interviewers assess whether you understand the performance implications of routine placement (package-level vs record-level operations) and whether you can distinguish appropriate use of start, end, and expert routines from ABAP that should have been a standard rule type. Be ready to explain how you would refactor a slow field routine and how routines interact with DTP error handling and semantic groups.