Types of trace/log generated by SAP system

The trace or log files are written by different SAP software components during different activities they are performing. They serve to purpose of revealing of errors that might occur. It is important to know where to find them and how to read them. In this blog post I focus on different types of trace/log that are generated by the SAP system.

In general we recognizes following traces within SAP:

·         System

·         Developer

In most cases the traces can be found in folder DIR_HOME of particular SAP instance. It usually points to “/usr/sap//DVEBMGS00/work” directory. Here there are files with naming convention stored:

DEVELOPER TRACES

These type of traces/logs are written by each kernel component (e.g. work processes, message server, gateway, etc.) and its depict all of their activities. Here are their types:

ABAP AS related log files

·         dev_w                   Work process traces, where n is work process ID

·         dev_wp                 Gateway work processes

·         dev_disp                       Dispatcher trace (ABAP)

·         dev_wdisp                    Trace of web dispatcher

·         dev_rd                         Gateway trace

·         dev_icm                       ICM trace

·         dev_ms                        Message server trace

·         dev_lg                         Local Gateway trace (LG-Layer, OSS connections to message server)

·         dev_cp                        Log of the SAP control program sapxpg

·         dev_xpg                       Log of all output from the external program

·         dev_evt                        External Event Raiser trace

·         dev_icf                        ICF (Internet Communication Framework) log file

·         dev_tp                        Transport control Program (tp) running using RFC logs (output of stdout)

·         dev_rfc                Developer Trace, where n is the number of the work process

·         dev_sapstart                Trace of startup phase (sapstart)

·         stderr<>                      Log of errors with sapstart process, during SAP ABAP AS start

o    stderr1: Info about the start process of database system.

o    stderr2: Info about the start process of message server

o    stderr3: Info about the start process of dispatcher

·         dev_st                         SAPGUI

·         vmc_w.log              SAP Virtual Machine Container (VMC) of JAVA AS, isolates individual users from each other 

·         vmc_w.trc              SAP Virtual Machine Container (VMC)

·         dev_moni                     Monitoring infrastructure trace

Enqueue server related

·         dev_enqadm                 Main enqueue server trace file

·         dev_eq*                       enqueue server trace, where n is workprocess ID

·         dev_enqlisten                Trace file of standalone enqueue server

·         dev_enqio_*                 Depending on the number of configured IO threads

·         dev_enqsig                   Log of processing of asynchronous signals from OS. Not on Win OS.

·         dev_enqsrv                  Trace is logged during start the enqueue server

·         dev_enqwork                Log of problems concerning actual enqueue processing

·         dev_enqrepl                  Log of communicates with replication server from thread

Replication server related

·         dev_enrepha                Log of events of the HA polling

·         dev_enrepsrv                Log of all components of enqueuer server except below ones.

·         dev_enrepsig                Log of processing of asynchronous signals of OS. Not on Win OS.

JAVA AS (J2EE) startup framework log files (restart)

·         dev_server             Java server stack trace

·         dev_server.out       Output to command line (output file)

·         dev_datcol*                  Snapshot generator for Java trace

·         dev_bootstrap               Trace of Application Server JAVA startup: Bootstrap phase

·         dev_bootstrap        as above

·         sdt_bootstrap.out output file of above

·         dev_deployment            Trace of Application Server JAVA startup: Deployment phase

·         dev_jstart                    Trace of Application Server JAVA startup: Start phase

·         dev_jcmon                   Trace of JCmon - native monitoring program

·         dev_sdm                      Software Development Manager for Java trace

·         sdt_sdm.out                 output file of above

·         dev_jcontrol                 trace of JControl framework which starts the bootstrap Java program

·         dev_dispatcher              Dispatcher trace (JAVA)

·         sdt_dispatcher.out         output file of above

In closing words of developer traces I just want to add that most of these files are available via tcode ST11 – Error log File.

SYSTEM TRACES

Manages in tcodes: ST05 – Performance trace or in ST01 – System trace. They are used to record internal SAP system activities. We recognize following system traces:

·         Authorization checks

·         Kernel functions

·         Kernel modules

·         Database accesses (SQL trace)

·         Table buffers

·         RFC calls

·         HTTP calls

·         Lock operations (client side)

Further information:

112 - Trace and error information in the "dev_" files

Wiki sys logs and traces

Development package for locked BW objects

In SAP BW version 74 SAP did a lot of work with respect to enhancing number of characters that particular object can have in its technical name. Also a much was done to enlarging object’s (characteristics) value from previously max 60 characters to 250/1333 characters (for more informacion see here). In brief what happened is that data element RSCHAVL which in pre 74 versions referred to domain RSCHAVL which had 60 chars (Data Type CHAR 60) in length is now referring to same domain but it has now (in 74) 1333 chars (Data Type SSTRING) in length.

There are of course many customers who developed their own custom functions based on old shorter version of domain/data element. Typical example is user exit based variables. Here customers are developing their own code to populate variable values. Since in pre 74 versions of BW the values could have only 60 chars in length and now it can have 1333 chars there might an issues pop-up running specific ABAP code. The code would expect 60 chars as maximum which is not the case in 74. SAP wants to support such customers and for compatibility reasons the “old style” of domain/data element are still available in 74 version. The old objects name is RSCHAVL60 for both domain and data element. What you need to do as part of BW74 upgrade you need to align such a code.

However this change has also an influence on where the domain/data element is assigned from software logistics perspective. It actually caused that data element object (RSCHAVL) was moved to other development package. The Dev pack where the data element belongs to is called RS_DTEL_LOCKED and that one belongs to superpackage BW_FOUNDATION.

As per this change it seems that the pack RS_DTEL_LOCKED is supposed to store objects that are not supposed to be changed anymore. Even it suggests its description as it can be observed in 74 based systems: “BI: Locked Data Elements”. All the objects in this package are then parked and their definition is not supposed to be changed.

ABAP Channels: ABAP Messaging and Push Channels

Traditionally interaction between user using e.g. SAP GUI and SAP system is like follows. User does some interaction (e.g. mouse click) and this is send to ABAP server to process. The ABAP server processes it and as per application logic it sends data back to client = SAP GUI where it is presented to the user (e.g. screen is refreshed with the newest data. This is what is called traditional SAP user interface design. It follows request/response model of interaction. However this is one of traditional obstacles in SAP systems. The traffic between client and server is very high. They basically exchange the data constantly and it also always requires user interaction (e.g. hitting refresh button). This approach is also called polling or event driven communication.

On other hand today’s application requires real-time data and interaction must be collaborative in way where user is not supposed to be inquiring information every time but as the information in application (e.g. in underlying database table) is changed but the user is supposed to get automatically instead. Means the application is not waiting for the user but it sends information (e.g. notifications) directly to the user instead of waiting for the request.

In SAP Application Server ABAP 7.40 Support Package 05 a new concept is introduced via infrastructure for event based communication. The new concept is called ABAP channels created in SAP NetWeaver Application Server ABAP. Now let’s take closer look how it is actually implemented.

SAP Web UIs such as WebGUI, Business Server Pages (BSP), Web Dynpro ABAP (WD4A) or SAPUI5 can use the ABAP Push Channel (APC) which enables such a real-time communication via WebSockets (over TCP/IP socket). The APC is bidirectional message based communication channel representing the WebSockets integration in ABAP. There is a possibility of push notifications to UI as soon as data change happened in the ABAP backend. This is a sign for a user to request the changed data from ABAP backend and update UI. The APC basically provides real-time UI access to ABAP’s data.

What we need in addition is to make interaction between events possible. Means event driven communication between ABAP sessions across different ABAP application servers. This is because the data might be processed by many sessions. So this would a way how the sessions are communicating between each other. Traditionally there would be a many polling performed to inquire the status of the data processing. Instead of this publish/subscribe mechanism is used to prompt the notifications. The ABAP Messaging Channel (AMC) infrastructure replaces the traditional polling. It acts as a broker for the messages exchange between different ABAP sessions (on different ABAP application servers). The AMC basically provides real-time communication between ABAP sessions.

Now how the two – APC and AMC plays together? By employing both of them scenarios when different users are working with same data of particular business object data. In here a changed data must be displayed on GUI of all users in the real-time without locking the object for changes by other users. With support of push channels, messaging channels and collaboration scenarios it is possible to use WebSockets to facilitate the exchange of messages between different ABAP sessions and enable users to collaborate over the infrastructure in real time.

Let’s go further. Imagine all possibilities that will come with Internet of Things (IoT). Having ABAP Messaging and Push Channels in place can enable many scenarios in management of shop floor and inventory areas be handled simultaneously.

Finally how is this done technically? It is implemented by Push Channel Protocol (PCP). It is SAP proprietary message based protocol. It is very similar to HTTP protocol. The PCP messages consist of header fields and a plain body. The body is separated by line feeds. Field values entries are in pairs of name-value separated by “:” sign. Implementation in the ABAP backend comprises of class CL_AC_MESSAGE_TYPE_PCP and interface IF_AC_MESSAGE_TYPE_PCP PCP which is having API. Implementation in SAPUI5 is within the class sap.ui.core.ws.SapPcpWebSocket. For other UI technologies there is a library sap-pcp-websocket.js for particular PCP methods.

More information:

Say goodbye to polling real time eventing in abap with abap channels

ABAP Push Channel (APC)

Introduction to abap channels

ABAP Channels Part 1: WebSocket Communication Using ABAP Push Channels

ABAP Channels Part 2: Publish/Subscribe Messaging Using ABAP Messaging Channels

ABAP Channels Part 3: Collaboration Scenario Using ABAP Messaging and ABAP Push Channels

Specification of the PCP

Deletion of spool requests

In case you see many SPOOL_INTERNAL_ERROR dump in your system you should done some action. In most of cases the dumps are occurring because there is no space left in the system. Therefore system can’t create new spool request as there is not enough space.

To clean it up you may want to delete old spool request that are not needed anymore. There is a tcode dedicated to this activity. Its name is SP01. You basically browse for the spool requests, mark them and delete in case you are sue you do not need them.

More information:

48284 - System cannot generate any more spool requests

SCN wiki

Searching for role by given BW objects

I was recently asked by user to analyze his authorization relevant issue. He wasn’t able to run the report. My user on other hand wasn’t able to run t-codes like PFCG. I also wasn’t sure what roles he has. My aim was just simply find all roles in the system which have particular cube (on top user’s report was built) assigned.

I needed to find out a table name which stored assignment of infocube names into Authorization Object (RSINFOCUBE) values. I thought that there would be the role where the cube is restricted.

After digging into the system I found my table. It is the table AGR_1251 - Authorization data for the activity group. Here into the field LOW I entered the cube name (ZMM*) which the report was based on. As an output I got role name (Z_BW_ADMIN_EU):

Normally this is assigned in t-code PFCG -> Authorization -> Display Authorization Data -> drilldown to particular Authorization Object. However as said already I wasn’t authorized to run t-code PFCG.

Disabling SSO in SAP app

Single Sing On (SSO) is a great thing. It saves us from slave labor of entering passwords into many different systems every time as we are already logged into trusted system (e.g. operating system) already.

However there are scenarios that we don’t need to be authenticated via SSO and we would like the system asks us for logon credentials. Most basic case is when we are testing something. And in such a cases it comes handy when we know how to suppress e.g. temporarily the SSO.

With regards to many SAP applications which are accessed by web browser there is a special parameter supplied that supports SSO disabling.  Usually name of the parameter is SPNEGO. It stands for Simple and Protected GSSAPI Negotiation Mechanism (SPNEGO). This thing actually serves as a protocol used to determine whether common GSSAPI (Generic Security Services Application Program Interface) mechanisms are available. If so it selects it and dispatches all security operations to it.

Whenever SPNEGO is used we are authenticated by SSO and there is no need to enter password again. The parameter is provided via URL of the application. How to use the parameter to disable the SSO?

1. SAP Portal usage:

http://hostname:port/irj/portal?spnego=disabled

2. NWBC usage:

          http://hostname:port/nwbc?spnego=disabled

3. example case for JAVA AS usage:

          http://hostname:port/nwa?spnego=disabled

http://hostname:port/useradmin?spnego=disabled

http://hostname:port/scheduler?spnego=disabled

4. example of common app usage:

http://hostname:port/my_app?spnego=disabled

Hiding ABAP code

For years ABAP had one special feature which not many people were aware of. Actually it was possible to hide ABAP source code of ABAP objects which were present in the systems and which were runnable in the system. Now you may wonder. Hold on what’s that? Well, we all know that ABAP code is a kind of open source for SAP customers as everybody who has an access and appropriate access rights in the SAP system can see the code. That’s correct. However there was a possibility to hide a code.

The trick is that by putting some special characters (in form of comment like: *@# ) into 1^st line of ABAP program in t-code like SE38 the whole code would disappear. Technically not whole code would disappear just its source code. The generated version stays in there and is still executable. Just for ABAP editors like t-code (SE38/SE80) it is invisible. While you would attempt to display it you would get following message instead of the code:

The source is protected. See explanation in long text

Message no. ED800

Diagnosis

The desired source text is protected by SAP because changing it could cause system errors.

Now why I keep writing this blog post using past time? Yep, this feature is gone now. As of SAP Kernel version 7.21 there is a fix delivered via SAP Note 2198580 which removed this feature. In fact the Note is disabling comment *@# completely.

More information:

2198580 - Code injection vulnerability in ABAP

ABAP web reporting URL parameters

Sometime back I wrote post about parameters of web reporting’s URL. That post was related to different parameters that we can employ into URL of web reporting based on JAVA or ABAP stack. In this post I want to list more parameter of ABAP based web reporting.

Recently I had to do stress tests of BW system. Most of my test cases involved web reporting. I need to quickly run multiple reports with deferent selection screen’s values. So I prepared simple HTML page where I put URL pointing to ABAP stack of the BW system. Each URL carried different parameter for BW report. Here are some of URL parameters I leveraged in my HTML page.

Parameters to address single value type of variable:

QUERY – mandatory parameter which tells the system which query to run

VAR_NAME_X – technical name of variable which is present on selection screen, where X is number to identity the variable

VAR_VALUE_EXT_X – value of variable on selection screen

Example of URL:

http://mybw.mydomain.com:8000/SAP/BW/BEX?&QUERY=Z_BW_QUERY&VAR_NAME_1=ZVAR1&VAR_VALUE_EXT_1=01.2014&VAR_NAME_2=ZVAR2&VAR_VALUE_EXT_2=01.2015

Parameters to address interval type of variable:

QUERY – see above

VAR_NAME_X - Technical name of the variable on report’s selection

VAR_VALUE_LOW_EXT_ X – "From" value for interval variable

VAR_VALUE_HIGH_EXT_X – "To" value for interval variable

Parameters to address select option type of variable:

QUERY – see above

VAR_NAME_X - see above

VAR_OPERATOR_X – Operator, following values are possible:

'EQ' = Individual value

'BT' = Interval

'LT' = Less than

'LE' = Less than or equal to

'GT' = Greater than

'GE' = Greater than or equal to

VAR_VALUE_LOW_EXT_ X - "From" value for select option variable

VAR_VALUE_HIGH_EXT_ X - "To" value for select option variable

VAR_SIGN_ X – in case it equals to 'I' values are included into the report’s output, in case 'E' found values are removed

Lastly I want to mention that as per my tests these parameters are valid for web reporting based on ABAP Stack of SAP BW.

More information:

Online docu: Transferring Variable Values

SAP BI web reporting URLs

Little help while implementing SAP Notes

Implementing SAP Notes is not an easy task. Although it is automated we can’t still talk about full automation of the task. There are still many tasks which need to be manually performed by someone who does the implementation. For some part I think there is no possibility to fully automate them.  However there are areas while SAP Notes processing which can be nicely automated.

Recently I read and blog post which address one of most painful parts. It is downloading the SAP Notes itself. We all know it. A bug in spotted in the system and SAP Note is found to fix it. Then someone (usually SAP Basis or ABAP programmer) is asked to do so. In case the Note is just one correction without any manual activities it is piece of cake. The Note is just simply downloaded in SNOTE (Note Assistant) tcode. Then it is implemented. Once it is tested it just needs to be confirmed in the SNOTE that implementation is okay.

However many times we are facing the SAP Notes that are having either many corrections, or they have may other SAP Notes as prerequisites or are having many manual steps. To address download of prerequisites Notes there is a tool that enables to upload multiple notes into SNOTE at one shot. You can easily download the Notes from SAP marketplace via Download Manager into local machine. Then via using this tool you can upload all the Notes form the workstation to SNOTE.

The tool is delivered via ABAP program ZSCWN_NOTES_UPLOAD - Load multiple notes files into SNOTE. More details about the tool see in blog: Report-zscwnnotesupload--load-multiple-notes-files-into-snote.

I found this tool very useful. I applaud Frank Buchholz who seems to be an author. Good job!

Personalizing of BW reports running in JAVA web reporting environment

Recently one user ran into strange issue with standard report. He was able to run the report on one day with predefined layout and on another day the layout was gone and he was not able to get it to original report layout. For all other users report looked like before in its original layout. The issue was only occurring in web reporting on top of JAVA stack. In case the user ran in ABAP based web reporting or in BEX Analyzer the report was just fine.

There were other strange things popping up while making an attempt to run the report. In case user tried to make right click on the report he got errors like following:

Error: Variable Entry No variables available Cancel

Error: Incorrect Output Type on Rendering Context Menu Snippets (!= OUTPUT)

Issue is caused by the fact that user somehow managed to activate personalization on that particular report. He was perhaps clicking too much into the report (nervous finger problem) and he accidently clicked on context menu where he choose item “Personalize Web Application” and sub item “Personalize”. This caused that layout that was changed by him before was saved and every time he accessed the report he was getting different report that original report’s layout.

The issue was resolved by deactivating the personalization. It was very hard to manage to click on the context menu and to choose “Delete Personalization” but finally we have done that and it resolved the issue.

You can only face this issue in the system where personalization of web reports is active. You can check that by running program: RS_PERS_ACTIVATE. The program also enables switching the personalization on and off. Here’s how the program looks like when also all other personalization types are active:

Core Data Services (CDS)

Within raising significance of SAP HANA there needed to be programming frameworks introduced to speed up accessing data in database to SAP applications. This is purpose of Core Data Services (CDS) which is an infrastructure for defining and consuming semantically rich and reusable data models. Modern SAP apps like SAP Business Suite or SAP S/4HANA are leveraging the CDS as it represents data layer for them. Framework around the CDS supports creation of reusable data models in database instead of the ABAP server. Similarly it supports consumption of these data models.

The data models are defined in ABAP programs as CDS views. By dealing with the CDS views the ABAP program has an access to the data in the data model. The view implements a semantic data model in the ABAP Dictionary. It allows to define entity types (business objects like customer, order, ...) and semantic relationships between these objects. The relationships correspond to foreign key relationships in traditional entity relationship (ER) models. Technically the CDS uses SQL-based DDL language to literally describe the relations. Moreover there are additions to DDL e.g. associations, which define the relationships between CDS views, and annotations, which direct the domain-specific use of CDS artifacts. By having described the semantics by the annotations we can use data and data types in different contexts (e.g. UI, analytics, searching).

The CDS was introduced within SPS 05 for SAP NetWeaver 7.4. In NetWeaver 7.5 there are more than 6k SAP standard view definitions that can be leveraged. This number of the CDSs keeps growing plus there more and more functionality being added in new SPS of NetWeaver 7.5.

More information

ABAP News for 7.40, SP08 - ABAP Core Data Services (CDS)

CDS wiki