Removing Duplicates and Building Unique Datasets

Explanation

Duplicate handling is common in reports, interfaces and data migration programs. The classic approach is SORT followed by DELETE ADJACENT DUPLICATES COMPARING fields. This works only when duplicate rows are adjacent, so sorting by the comparison fields is important. For unique-key datasets, a HASHED table can prevent duplicates by design. In Open SQL scenarios, unique driver tables are important before FOR ALL ENTRIES or key-based data fetches. Duplicate handling is not just a technical cleanup step; it affects business correctness. Removing duplicates by the wrong key can remove valid business records.

Code example

DATA lt_matnr TYPE STANDARD TABLE OF matnr. LOOP AT lt_vbrp INTO DATA(ls_vbrp).  APPEND ls_vbrp-matnr TO lt_matnr.ENDLOOP. SORT lt_matnr BY table_line.DELETE ADJACENT DUPLICATES FROM lt_matnr COMPARING table_line.

Real project scenario

Before fetching material descriptions for sales order items, build a unique material driver table. This reduces database input size and avoids repeated processing of the same material.

Common mistakes

- Using DELETE ADJACENT DUPLICATES without SORT. - Comparing wrong fields. - Removing duplicates when business requires duplicate records. - Building driver tables with blank key values.

Best practices

- Sort by comparison fields before deleting duplicates. - Understand business key before removing duplicates. - Use HASHED table when uniqueness is required by design. - Remove blank keys before using driver tables.

Interview angle

A practical answer should mention SORT, COMPARING fields and business key correctness.