BW 7.x: How to design start/end/expert routine within transformations?

There are several points that you need to recon while coding start, end or expert routines. 

Basically you need to be aware which data are you processing either it is source_package in case of start routine; result_package in case of end routine or even both tables in case of expert routine. Basically you do LOOP ABAP statement over one of those internal tables and you modify it or add new rows. There are available corresponding work areas (source_field, result_field) and field symbols (, ) that make you looping easier. 

Another point to be considered is creating new records within routines in one of its data_package internal tables. Here’s field called RECORD which needs special treatments. In case of end routine you need to populate value of this field by calling standard method new_record__end_routine.

In case of error handling you can populate called MONITOR. By this you can send messages to the Data Transfer Process (DTP) monitor. This makes maintenance of BW application easier. BW administrator sees error in monitor and can recognize its root cause directly by analyzing message instead of digging in coding or in ABAP dumps. Coding can be maintained like following:

IF sy-subrc <> 0.

  monitor_rec-msgty = 'msgty'. "e.g. W

  monitor_rec-msgid = 'msgcls'. "e.g. RS

  monitor_rec-msgno = '100'.

  monitor_rec-msgv1 = lv_par1. "Parameter 1

  monitor_rec-msgv1 = lv_par2. "Parameter 2

  monitor_rec-msgv1 = lv_par3. "Parameter 3

  monitor_rec-msgv1 = lv_par4. "Parameter 4

  APPEND monitor_rec TO MONITOR.

  RAISE EXCEPTION TYPE CX_RSROUT_ABORT. "aborts whole processing
*  RAISE EXCEPTION TYPE CX_RSROUT_SKIP_RECORD. "skips this particular record

ENDIF.

For  further reference see following SAP Notes:

1227667 - Guidelines for expert routine: Design rules
1223532 - Design rules: Adding records to end routine
1258089 - Design rule: Adding records to the start routine

- update 21.01.2011 - 
In case of expert routine, breaking processing of load via raising of exception doesn't work (CX_RSROUT_ABORT). You have to use method send_message of class cl_rstran_expert_rout_srv.

- update 19.05.2011 - 
Some more SAP Notes on this topic:
1349820 - Expert routine: Technical enhancements

Limitations of DTP’s filter

Recently I encounter interesting behavior regarding amount of entries that DTP’s filter is able to populate. I created ABAP routine to fill up entries in filter. Since my routine produced quite big portion of data; I was curious how much entries is system capable to process.

For first run my routine produced over 1mil records. However data load in general was not successful. It was set to red with following message BRAIN299:

I still have an impression that load could be successful since I compared the data entries produced in filter and apparently they all were processed correctly. So I assume load would be OK.

Afterwards I tried to lower number of entries produced by my DTP Filter routine. My intention was to get number of entries which system is able to process without this error. I succeeded with 21.001 entries. All higher entries like 21.002 ended again with error mentioned above.

There is following note available on OSS:

1077708 - 7.0 Support Package 16 DTP filter restricted to 10,000 lines

According this Note seems that this restriction was even lower in past. Till SP#16 restriction was 10.000 records. Since I did my testing in system based on SP#20; here’s restriction is 21.001 records. Would be interesting to check either this restriction is removed in newest version 7.3 of BW.