ABAP Developmentadvanced

Internal Tables Part 10: Memory Management and Large Internal Tables

Key takeaway

Process large data volumes without memory dumps: estimate table size, select only needed columns, process in packages, free memory with FREE, avoid unnecessary copies, and analyze memory with the debugger and memory inspector.

Explanation

📌 KEY POINTS
✅ Memory ≈ rows × row width — select only the columns you need.
✅ Process huge volumes in packages (SELECT … PACKAGE SIZE) instead of loading everything.
✅ CLEAR/REFRESH empties a table but keeps memory; FREE releases it.
⚠️ Dump TSV_TNEW_PAGE_ALLOC_FAILED means the program exceeded its memory quota — design, don't just increase limits.

━━━━━━━━━━━━━━━━━━━━
▶ WHERE WE ARE
Parts 1–9 made table processing correct and fast. With millions of rows, memory becomes the limit.

━━━━━━━━━━━━━━━━━━━━
▶ 1. HOW BIG IS MY TABLE?

Row width = sum of field lengths (SBOOK * is much wider than 5 selected fields).
5 million rows × 200 bytes ≈ 1 GB — plus copies.
Check in the debugger: table view shows rows; memory analysis shows bound memory.

━━━━━━━━━━━━━━━━━━━━
▶ 2. SELECT ONLY WHAT YOU NEED

Field list instead of SELECT *.
WHERE conditions instead of filtering in ABAP.
Aggregate in the database (SUM, COUNT, GROUP BY) instead of reading all rows to sum them.

━━━━━━━━━━━━━━━━━━━━
▶ 3. PACKAGE PROCESSING
SELECT … INTO TABLE gt_book PACKAGE SIZE 10000.
process gt_book (write file, post, aggregate)

ENDSELECT.

Only one package is in memory at a time. Each iteration replaces the previous content (use APPENDING TABLE to accumulate — which defeats the purpose). Details: Open SQL Part 7.

━━━━━━━━━━━━━━━━━━━━
▶ 4. AVOID COPIES

Assigning a table to another variable shares memory until one is changed (copy-on-write) — then a full copy is made.
Passing tables to methods by reference (default for IMPORTING) avoids copies; VALUE( ) parameters copy.
Loop with ASSIGNING instead of INTO for wide rows.

━━━━━━━━━━━━━━━━━━━━
▶ 5. RELEASE MEMORY

CLEAR gt_x / REFRESH gt_x — rows deleted, memory kept for reuse.
FREE gt_x — rows deleted and memory released. Use after large intermediate tables are no longer needed.

━━━━━━━━━━━━━━━━━━━━
▶ 6. TOOLS

ST22 — memory dumps (TSV_TNEW_PAGE_ALLOC_FAILED, SYSTEM_NO_ROLL) with the table that grew.
Debugger → Memory Analysis / Memory Inspector (S_MEMORY_INSPECTOR) — which objects use memory.
SAT for runtime, including memory hotspots.

━━━━━━━━━━━━━━━━━━━━
▶ STEP BY STEP

Compare the memory of SELECT * vs a field list for SBOOK (debugger memory view).
Rewrite a full load as package processing (code example).
FREE intermediate tables after use.

✅ CHECKPOINT: your programs scale to production volumes. Internal Tables complete — next topic: Open SQL.

Code example

ABAP Code

REPORT zitab_part10. TYPES: BEGIN OF ty_book_small,          " only needed columns         carrid   TYPE s_carr_id,         connid   TYPE s_conn_id,         fldate   TYPE s_date,         forcuram TYPE s_f_cur_pr,         forcurkey TYPE s_curr,       END OF ty_book_small,       BEGIN OF ty_total,         carrid    TYPE s_carr_id,         forcurkey TYPE s_curr,         amount    TYPE s_f_cur_pr,       END OF ty_total. DATA: gt_package TYPE STANDARD TABLE OF ty_book_small,      gs_book    TYPE ty_book_small,      gt_total   TYPE HASHED TABLE OF ty_total WITH UNIQUE KEY carrid forcurkey,      gs_total   TYPE ty_total. FIELD-SYMBOLS <fs_book> TYPE ty_book_small. START-OF-SELECTION.* ===== BAD: everything, all columns, at once =====* SELECT * FROM sbook INTO TABLE gt_all.     -> may dump with TSV_TNEW_PAGE_ALLOC_FAILED * ===== GOOD: needed columns, packages of 10,000 rows =====  SELECT carrid connid fldate forcuram forcurkey    FROM sbook    INTO TABLE gt_package    PACKAGE SIZE 10000    WHERE cancelled = space.     LOOP AT gt_package ASSIGNING <fs_book>.          " no copy of wide rows      gs_total-carrid    = <fs_book>-carrid.      gs_total-forcurkey = <fs_book>-forcurkey.      gs_total-amount    = <fs_book>-forcuram.      COLLECT gs_total INTO gt_total.               " small result table    ENDLOOP.  ENDSELECT.                                         " next package replaces gt_package   FREE gt_package.                                   " release memory of the buffer table   LOOP AT gt_total INTO gs_total.    WRITE: / gs_total-carrid, gs_total-amount, gs_total-forcurkey.  ENDLOOP. * ===== EVEN BETTER when only totals are needed: aggregate in the DB =====* SELECT carrid forcurkey SUM( forcuram ) FROM sbook*   INTO TABLE gt_total WHERE cancelled = space*   GROUP BY carrid forcurkey.

Real project scenario

A year-end archiving preparation program selected all fields of 30 million accounting line items into one internal table and dumped with TSV_TNEW_PAGE_ALLOC_FAILED. Basis was asked to raise memory quotas, which only moved the dump later.
The redesign selected 8 needed fields, processed in packages of 50,000 rows, and wrote results to a file per package. Memory usage stayed under 200 MB and the program completed in a normal background job.

Common mistakes

SELECT * into huge tables.
Filtering and summing in ABAP instead of SQL.
Accumulating packages with APPENDING TABLE.
Copying large tables unnecessarily (VALUE parameters, modifications of shared copies).
Asking for bigger memory quotas instead of fixing the design.

Best practices

Select only needed columns and rows.
Aggregate in the database where possible.
Use PACKAGE SIZE for mass processing.
Use FREE for large intermediate tables.
Analyze memory with the debugger and memory inspector.

Interview angle

Questions: "How do you avoid memory dumps with large internal tables?", "CLEAR vs REFRESH vs FREE?", "What is PACKAGE SIZE?"
Strong answer: estimate memory as rows × width, select only needed columns/rows, aggregate in SQL, process with SELECT … PACKAGE SIZE (one package in memory), avoid copies (reference passing, ASSIGNING), FREE large tables when done; CLEAR/REFRESH keep allocated memory, FREE releases it; analyze with ST22, debugger memory analysis and memory inspector.

Practice this lesson

Quick check

1. Which statement releases the memory of a large internal table?

2. A program reads 30 million rows into one internal table and dumps with TSV_TNEW_PAGE_ALLOC_FAILED. What is the best fix?

3. A structure contains an internal table as one of its components (a nested/deep table). When this structure is assigned to another structure of the same type using the assignment operator =, what happens to the nested table component?

Interview questions

1. A background job processes a large internal table and occasionally dumps with a short dump related to memory exhaustion, but only when run against certain company codes with more data. What internal-table-specific patterns would you look for in the code as likely culprits?

Answer it in your own words first, then compare.

2. Explain the memory and performance implications of internal table choices in large background jobs.