SAP Change Data Capture (CDC)

Change Data Capture (CDC) allows identifying and capturing changes made to data within SAP application tables with the SAP system. Those data changes can be then in real-time replicated to other systems or processes. It's a way to keep data synchronized across different environments, enabling real-time analytics, data warehousing, and other data-driven initiatives.

There are different techniques for implementing CDC in SAP environments:

Log-Based CDC: This approach reads the transaction logs of the SAP database to identify changes. It's generally considered the most efficient method with minimal impact on the source system.

Trigger-Based CDC: This method uses database triggers to capture changes as they occur. Triggers are database objects that automatically execute a predefined action in response to a specific event (e.g., an insert, update, or delete operation).

Table-Based CDC: This technique involves comparing snapshots of tables at different points in time to identify changes. It's less efficient than log-based or trigger-based methods but can be used in situations where those options are not available.

SAP has several tools and technologies that leveraging CDC, including:

SAP Data Services: A data integration and data quality platform that includes CDC capabilities.

SAP Landscape Transformation (SLT) Replication Server: A tool for real-time data replication from SAP systems to SAP HANA.

There are many 3^rd party tools that uses SAP CDC as well.

Now when it comes to extracting the data via CDS (Core Data Services) aka ABAP CDS views the trigger-based CDC is used as well. Specific annotation for the CDS view needs to be specified in ABAP Development Tool (ADT Tool). The annotations allow the CDS view to use a trigger-based CDC recording mechanism to load data and record changes to the tables that belong to the view.

PS: Do not confuse SAP CDC (Change Data Capture) with other SAP products like SAP Customer Data Cloud.

Mor information:

Online documentation

Exposing SAP data with a Note 3255746 in mind

In a nutshell with an update of the Note 3255746 SAP stops customers and 3^rd party applications from using RFC modules in the ODP Data Replication API to access and extract SAP data from ABAP sources. As these APIs are only for SAP’s internal use. This is an SAP's ban of RFC usage.

This means it is not possible to use ODP to access ABAP CDS views directly. Instead, the ODP component with OData API provided by SAP for data extraction should be used. This API is stable and recommended for all customer and third-party applications. Furthermore, SAP advises customers “to use SAP Datasphere for realizing data replication scenarios to move data from various SAP sources (such as SAP S/4HANA, SAP BW, SAP ECC sources etc.) into third-party applications & tools.”

Restriction was introduced on February 2nd, 2024, when SAP updated the Note mentioned. Note originated sometime around year 2022 when it stated that “ODP RFC method were unsupported”. But 2024 Feb update of the Note made unsupported -> unpermitted.

Impact on 3rd party tools / Target platforms

SNP/DataVard Glue - no impact as the tool uses an technology that is independent of any third-party (SAP's in this case). More info.

TheoBald Xtract – impacted, Theobald claims it will enhance its tool to use OData API (as well as it is currently utilizing the RFC modules) sometime around 2024.Q4. More info.

Init Software – Init ODP Source Connector is impacted. Use Init OData v2 Source / Sink Connector More info.

Databricks – impacted, as their lakehouse ingests data from many sources More info.

Azure Data Factory (ADF) / Azure Synapse Analytics - data ingestion platforms of MS Azure is impacted by this as Azure Data Factory SAP CDC connector is using the ODP framework. Use OData API based connector instead that is available in Azure. More info.

Qlik Replica – impacted, More info.

Google Cloud Data Fusion – As there are many GCP connectors for SAP (e.g. BigQuery, SAP BW Open Hub Batch Source, SAP OData, SAP ODP, SAP SLT Replication, SAP Table Batch Source) some of them are impacted. Specific case needs to be reviewed and a new integration needs to be built based on OData API if impacted. More info.

AWS Appflow – There may be a several connectors available in AWS Appflow. In case of “Amazon AppFlow SAP OData connector” usage there is no impact as it leverages OData API.

HVR/Fivetran - Fivetran's SAP NetWeaver connector uses RFC calls thus it is impacted. Most likely there will be a new Fivetran ODP connector released soon to be based on ODP OData API. More info.

Snowflake – depends on which tool is used to replicate the data to the snowflake.

Notice that there are many other tools used to expose SAP data that are not listed in here.

Also, the big question if this action done by SAP is legally binding its customers. Well, it is a big topic actually. What is driving it is the SAP Software Use Rights signed. There is a standard version of it (but also notice that the document is a subject to evolution and depends on the SAP version, etc.) that clearly says that asynchronous indirect access to SAP data is not licensed without SAP Digital Access, BW or OpenHub. Thus, one can believe that even SAP puts auditing/tracking mechanisms in place and they prove by that that particular customer is violating the Note 3255746 they can’t do much about it. But again, a disclaimer – this is not a legal advice at all! Always consult with your SAP account manager also reach out for professional help of e.g. companies providing SAP licenses consulting.

 

More information:

3255746 - Unpermitted usage of ODP Data Replication APIs

ODP-Based Data Extraction via OData

Guidance for Partners on certifying their data integration offerings with SAP Solutions