No Routine (ROUT) objects in BW4 transport of transformations

You may notice that in BW4 based systems transporting of Transformation is simplified. In classic versions of BW there were usually objects like Routine included in the transport requests in case particular Transformation contained them. The objects (OBJECT = ROUT) represent either formula in transformation rules or Start/End/Expert Routine. Such transport requests look like below one:

Routine objects of the Transformations are stored in table RSTRANSTEPROUT (Rule Type: Routine). They still are there in the table in case of BW4 systems.

However, in BW/4 based systems there are no transport entries like that anymore. Reason is that, routines and formulas are embedded with transformation metadata. The table RSTRANSTEPROUT has a column CODE that contains all the code of a particular routine and formula. Because of that the ROUT object are not necessary anymore. This change is in place since SAP BW/HANA 1.0 SP04.

 

More information:

2548884 - Formula handling and transport related problems in BW/4HANA

SAP GUI 8

SAP released a new version of SAP GUI for Windows – version 8 on 20 Jan 2023. It is the first version that is 64-bit version that is delivered along 32-bit version. While installing the version 8 on WINDOWS OS 64-bit user needs to decide which version of the GUI will be installed. Parallel install of both versions the 32 and 64 bits is not possible. What are other new features of the new version of SAP GUI? Here I’m mentioning just a subset of new features that I find interesting.

UI point of view:

Themes:

1. Quartz theme got new rendering engine

Usability:

1. Changes to combo box control and the "normal" combo box

2. Tree control has a scrolling indicator now. This will be useful while navigating in BW’s RSA t-code e.g. in Info Object or Process Chain trees. But who is now using the SAP GUI for BW? We all moved to Eclipse based SAP HANA Studio with BW modeling tools, right? :-)

3. Table Control and Dialog box Container supports additional hot keys to enable faster navigation, e.g. CTRL + UP/DOWN/HOME/END.

4. ALV grid support a paste with gaps feature.

5. Branding support: selected branding image per SAP system and client can be defined now.

Technical point of view:

HTML Control based on WebView2 (package that enables to render web content via MS Edge browse rendering engine) supports a browser tabs, which were missing in SAP GUI 77 via browser extension from the Microsoft Edge Add-Ons.

 

More information:

Online docu

3075781 - New and changed features in SAP GUI for Windows 8.00

3218166 - SAP GUI for Windows: Functional differences of the 64-bit version compared to the 32-bit version

2035293 - Known and open issues of SAP GUI for Windows

Package based aDSO activation

In SAP BW, aDSO (advanced DataStore Object) or DSO (DataStore Object) is used as a persistence layer for storing and processing data. In case of e.g. standard DSO a data activation is the process of moving the data from inbound table of the DSO to active table. Data is aggregated accordingly within this process. See Flavors of aDSO object for more details.

In case there is a huge number (10k+) of data loads requests in specific aDSO objects and they all are being activated together there can be problems. Mostly with the respect to memory issues.

This is solved in the newer BW releases (7.50, BW/4HANA 1.0, BW/4HANA 2.0, BW/4HANA 2021) while introducing so called package based activation. This type of the data activation is available when the activation runs over Process chain – by process variant "Clean Up Old Requests in DataStore Objects (Advanced)". Here in this process s package size can be defined for specific aDSO object by a user. If it is not defined, a default package size of 10k is used.

The value itself is stored in table RSPCVARIANT (Generic Variant Storage):

The new way of the package based data activation only works when it is triggered from the in process chains via above mentioned process variant. In case the activation is performed manually, either via t-code RSMNG or via BW/4 Cockpit the activation work in the old way – no package size used.

Technically the activation is performed by a method _OPEN_PROCESS_ACTIVATE of class CL_RSDSO_MNG_ACTION / CL_RSDSO_DATA_TASK) triggered by IF_RSPM_RUNTIME~ACTIVATE_PROCESS (CL_RSPM_RUNTIME) ) and/or FM RSDSO_COMPRESS_REQUESTS.

 

More information:

3108217 - ADSO: packaged activation of requests

3038366 - Activation of requests in an aDSO with a huge number of request fails

Flavors of aDSO object

Watermarks of a BW data target object

Watermarks in BW can be considered as set of an internal counters (so called watermarks) of a technical information of the BW data target. Following are the BW data target objects that may have the watermarks: InfoCube, DataStore Object, Master Data Table and Text Table.

There is an ABAP program (RSPM_ADSO_WATERMARKS) in SAP BW systems that displays the aDSO watermarks like:

·        Number of all requests in aDSO

·        Number of nonactive requests in aDSO

·        Number of deleted requests in aDSO

·        TSN of lowest active request (AQ) in aDSO

·        TSN of high
est active request (AQ) in aDSO

·        AQ Status

·        TSN of lowest active request (AT) in aDSO

·        TSN of highest active request (AT) in aDSO

·        AT Status

·        Count

·        Count AT

·        Data in aDSO must be activated

·        Proc. Type

·        Process ID

·        Datamarted source TSN

·        Requests not datamarted

·        Delta information

 

The program gathers information from many tables like RSOADSO, RSPMREQUEST, RSMDATASTATE_DMO (DMO - DataMart Out), RSMDATASTATE_DMI (DMI - DataMart In) and others. For non BW4 optimized objects like classic infocube there is an function module RSSM_SHOW_WATERMARKS that gathers the watermarks for those objects. 

Selection screen of the report RSPM_ADSO_WATERMARKS:

Example output screen of the same report:

Reassigning of Process Chain’s InfoArea

When I needed to reassign a process chain (PC) between infoareas (IA) I was always struggling. Only way, how to do it I knew; was a drag and drop type of thing in backend t-codes like RSA1, RSPC1, etc. In this way, I moved my PC within the IA’s tree from current IA node to new one. In case a distance between the old and new node was quite big I ended up with moving the mouse until I got into the new IA. And sometimes it took even couple of minutes to get there.

However, with an introduction of BW Cockpit the PC’s IA reassignment gets a lot more easier. There a functionality built in the BW Cockpit to do it very easily. While in Process Chain Editor part of the BW Cockpit there is a Properties button on top right part of the page.

The button shows up a dialog window where an InfoArea can be easily changed to new one among other attributes of the PC. The information of the assigned IA is saved in table RSPCCHAINATTR and column APPLNM.

More information:

How to find out InfoArea for particular BW object type – in BW/4 systems?

BW’s InfoArea vs Application components

BW/4HANA Cockpit

Other posts on Process Chains topic

SAP Datasphere is the new SAP DataWarehouse Cloud

This week on March 8^th 2023 the during a SAP Data Unleashed event SAP announced a new solution called SAP Datasphere. What it means is that Datawarehouse Cloud (DWC) becomes SAP Datasphere.

 

What led SAP to announce this solution? Basically, they are trying to address today’s challenges of data architecture: from data warehousing (structured data) to data lakes (unstructured or any kind of data) and beyond to data fabric (integrated layer (fabric) of data and connecting processes) reaching to particular challenges like data federation, cataloging, lineage, metadata, integration and semantic modeling of data.

How does the SAP address these kind of challenges it? By mixing a portfolio of their existing products like:

DWC – Data warehouse solution in cloud that evolved from SAP BW (BW/4), customers can move their BW models to Datasphere/DWC via SAP BW Bridge (BWB), thus investments made into BW are safe.

SAP Analytics Cloud (SAC) – Datasphere is integrated into SAC by supporting its analytics and planning use cases.

SAP Data Intelligence Cloud (SAP Data Intelligence formerly SAP Data Hub) – Datasphere leverages its Data Catalog functionality and engines for data moving.

 

And in addition, solutions from their partners like below to support the Business Data Fabric:

Databricks – provides data lakes platform called lakehouse (data warehouse + data lake) initially based around Apache Spark.

Confluent – capturing data in motion capabilities based on Apache Kafka.

Collibra – data governance and metadata management capabilities.

DataRobot – capabilities of AI lifecycle management, a platform for augmented intelligence – AutoML.

 

All these capabilities together forms the Datasphere. Although technically speaking it is a combination of DWC and SAP Data Intelligence Cloud. The Datawarehouse Cloud is rebranded to the Datasphere claiming the Datasphere to be a next generation of the DWC. Simple speaking features of the data integration, data cataloging, and semantic modeling were added into the DWC to enhance its data discovery, modeling, and distribution capabilities making it the Datasphere.

Data can be either replicated into the Datasphere or federated. SAP emphasizes an approach of data can sit anywhere just its analytics runs in the Datasphere. This is crucial point as running the data warehouse in the cloud may not be scalable easily. Thus, keeping data in its source and not replicating it may sound a better options.

User of the Datasphere digs into so called Datasphere Catalog to find a data of his/her interest. Leveraging its lineage capability a relationships between the different data can be explored. The Catalog supports data objects from other SAP Datasphere instances and SAP Analytics Cloud. This should be expanded soon to other SAP apps (like BW, ECC, S/4) plus non SAP apps via its partners. While accessing data like this the data from one source can be enhanced/mixed with data from other sources just by the user in Datasphere Spaces. Assuming here that the spaces are next generation of BW workspaces.

 

My take

Nowadays organizations are processing data outside their enterprise systems more and more. Therefore, a solution to enable users to work in analytics area using data from “anywhere” is very plausible. This is not a new for SAP. Somewhat similar picture was painted when Data Intelligence/Data Hub came on board. Seeing the Datasphere as a successor to those initiatives (DI, BW, DWC) plus having its AI powered capabilities the idea of the business data fabric perhaps may come true in future having a kind of “chatGPT” style of analytics.

 

More information:

SAP DataSphere microsite

onlinedocu

SAP Data Unleashed event

SAP Data Warehouse Cloud (DWC), SAP BW Bridge (BWB)

What are types of ABAP development packages?

Development package serves for purposes of structuring ABAP development objects into logical units.  In addition to organizing of the ABAP objects while the development packages can be organized in other packages it enables SAP software logistics to function.

Organizing of packages within other packages is enabled by the type of the development package. There are following three types (as per domain MAINPACK):

'S' - Structure packages

'X' - Main packages

'' - Development packages

 

A purpose of the structure packages is to act as top level container in the package hierarchy that defines architecture of the objects within the subordinate packages. This type of the package does not contain any development objects itself. Instead, they can contain package interfaces and subpackages.

A purpose of the main packages can be seen as group of semantically similar objects. As well as the structure packages the main one do not contain any development objects itself as well.

Development package groups development objects itself as per their types (e.g. domains, data elements, programs, includes, classes, etc.)

Main packages, with the exception of your own package interfaces and subpackages, cannot contain other repository objects. Subpackages, in turn, can be other main packages and the standard development packages.

Package are being created in t-code SE21 Package Builder. All the packages are stored in table TDEVC.

 

What development packages can be seen in the SAP NetWeaver/ABAP Platform based systems?

$TMP – Local or temporary objects, most visible one. All objects that are not needed to be transported into other system go in here.

Z* or Y* – custom development packages. All objects created by customer are stored in those packages.

$HV – Generated Help View Program

$MC – Generated Matchcode Programs/Functions

$ENQ – Generated ENQUEUE Function Modules

$GEN – Other generated objects

$SWF_RUN_CNT – Workflow Container: Generated Data Types

$EQ_GEN – Stores BW Easy Query generated objects. The package is created automatically upon first easy query generation. It includes all generated BW Easy Query objects such as the function modules.

 

More information:

ABAP Packages

SAP glossary

BW Background Job - Cleanup of orphaned DTIS entries

DTP may run in DTIS (Data Transfer Intermediate Storage) mode. That mode works in a way that extracted data is stored in in intermediate storage before they are updated to target.

The Intermediate Storage contains many records even though the corresponding DTP requests are already deleted or are in green status without any error. Thus the intermediate storage can grow up fast and it can be a problem while having many DTPs in the BW system running in that mode.

There is an ABAP program RSDTIS_CLEANUP that can run as a job to clean up the Intermediate Storage of DTIS DTPs that belong to deleted requests.

The report can run for specific DTP or for multiple DTPs that are identified either by source object (aDSO, DataSource, Query Element, InfoSOurce, Composite Provider or InfoObject) or by Unique DTIS Number. The Unique DTIS Number (RSDTIS_NUMBER) is a sequential number that belong to specific execution of and DTP in the DITS mode. All the Unique DTIS Number are stored in table RSDTIS.

More over there is a checkbox that enables a deletion of unused DTIS tables.

More information:

2708942 - Cleanup of orphaned DTIS entries

SAP BW DTIS - Data Transfer Intermediate Storage

Collections of BW Background Jobs

SAP BW (Business Warehouse) background jobs are automated tasks that are scheduled to run at regular intervals without requiring manual intervention. These jobs are used for various purposes in the BW system, including data loading, data transformation, and system maintenance. Regular SAP BW background jobs are essential for ensuring the efficient operation of the BW system. They help to automate repetitive tasks, reduce the risk of human error, and improve system performance. They can also help to ensure that critical data is available to end-users when needed.

Here are some of the most common types of regular SAP BW background jobs:

Data loading jobs: Used to load data from source systems into the BW system. They may include jobs that extract data from external systems, jobs that transform data into the correct format for BW, and jobs that load data into BW data targets.

Data transformation jobs: Used to transform data from one format to another. They may include jobs that perform data cleansing, data validation, or data enrichment.

System maintenance jobs: Used to maintain the health and performance of the BW system. They may include jobs that perform database maintenance, system backups, or system health checks.

Archiving jobs: Used to archive data that is no longer needed in the active BW system. Archiving helps to reduce the size of the active BW system, which can improve system performance.

Broadcasting jobs: Used to generate reports based on data stored in the BW system.

Here’s a list of my blog posts related regular background jobs in BW:

1. Cleanup of orphaned DTIS entries

2. Deletion of Orphaned Entries in Errorstack/Log Job RSBKCLEANUPBUFFER

3. DTP Clean Up Job RSBKCHECKBUFFER

4. Job BI_WRITE_PROT_TO_APPLLOG

5. Deletion of BW Messages/Parameters

SAP Profitability and Performance Management

It is SAP solution designed to analyze financial performance and profitability of an organizations. It provides advanced analytical tools and reporting capabilities to help businesses identify the factors that drive their profits and optimize their performance. It is also known as SAP Performance Management for Financial Services or SAP Profitability and Performance Management Cloud or as SAP PaPM that is abbreviation of Profitability and Performance Management. Sometimes it is also referred as NXI (software component) as it is being developed by partner company called NEXONTIS (part of msg global solutions ag).

 

More information:

PaPM microsite

Help site

BW Background Job - Deletion of BW Messages/Parameters

BW background management provides basic functions for functions for managing background and parallel processes in the BW system. Collection of the background management plus logs and tools is what is BW data processing all about. Simply speaking it comprises of all the tasks that runs as the background jobs that are performing BW tasks like data loading, parallelization, indexing, filling aggregates, archiving, attribute change run, broadcasting, compression, data activation, rollups, remodeling, collecting statistics, logs, messages, etc. All the BW background management jobs are visible in t-code RSBATCH.

All these tasks running all the time in the BW systems are producing a great amount of logs and messages. The table that is used to store those logs and message is RSBATCHDATA. In order to prevent the table to grow a lot there is a job that can do a housekeeping activities. The job is usually called BI_DELETE_OLD_MSG_PARM_DTPTEMP and it runs the ABAP program RSBATCH_DEL_MSG_PARM_DTPTEMP. It deletes old messages, parameters and temporary DTP Data etc. On its selection screen, it is possible to specify the no of days as a retention period (delete all records older than X days, where X is from zero to 999) for messages, logs and DTP temporary data. It usually runs on daily basis.

The job can be scheduled from t-code RSBATCH -> under Settings and Parallel Processing part -> Deletion selection.

More information:

t-code RSBATCH

Number of Processes for DTP Parallel Processing