Introduction
You probably have heard before that one of the strongest muscles that SAP Datasphere brings to the table is that it leverages, respects and maintains SAP business context from S/4HANA’s semantic layer.
As Business Context I refer to: Metadata, Hierarchies, Currency Conversions, Unit of Measures, Multi-Language, Security Configurations; in other words, all objects associations that give business meaning to the data.
As an example: let’s picture a simple yet common use case in supply chain analytics: you would like to understand the impact of discount negotiations you have had with your vendors/business partners in your procurement processes. Connections amongst your business partners (master data) and their related purchase orders, currency rates across countries (currency conversions), where they operate and in what levels, like international, national and regional geographic-wise (hierarchies) -just to mention few examples- act as the “glue” (business context) that gives business meaning on the “raw” data.
In my previous blog, I talk about the great flexibility that SAP Datasphere offers for data integration to enable Business Data Fabric Architectures. Especially when leveraging data federation mechanisms to reduce storage costs and copies of the data. However, it is known that depending on the use case and performance on the usage of high volumes of data, data replication might be the best option.
S/4HANA plays a key role in this journey, as the ERP Cloud core system, S/4HANA combines the power of transactions and analytical engine to expose enterprise data through virtual data models that are consumed with Core Data Service (CDS) views.
Additional note: SAP S/4HANA provides over 8000 CDS views where around 20% are delta capable. These CDS views are out-of-the box available for Data Extraction, and the ones that are not, can be data-extraction enabled by creating custom CDS views which retrieve the respective associations and semantic relationships they have with other views and objects.
Check this awesome blog from @FrankRiesner for more information about released data extraction enabled CDS views per S/4HANA version.
Let’s see know why Datasphere is the spot on solution to maintain the business context of your SAP Data.
Semantic Onboarding – Leverage your SAP Data in Datasphere
Whether your SAP data consumption is through federation or replication, the Semantic Onboarding capability is an excellent way to provide visibility to technical and business data users of all released CDS views and their relationships/associations with other objects. The way that it works is that you actually import SAP semantics to Datasphere, enabling easy and earlier access to those views for data modeling and ultimately analytics consumption.
SAP Datasphere provides mechanisms to maintain the semantics of the SAP data for many business context objects such as the ones I mentioned above: Metadata, Hierarchies, Currency Conversions, Unit of Measures, Multi-Language, Security Configurations.
The Semantic Onboarding supports different source SAP systems, such as S/4HANA Cloud, S/4HANA, BW/4HANA, the BW Bridge and soon calculation views from HANA Cloud will also be supported, see the roadmap item.
Let’s see a quick example of how the Semantic Onboarding looks and works. Here, I’m leveraging a connection to an S/4HANA Cloud system and importing the I_Product CDS View, to later see it available as a table in the Data Builder, where data analysts can start working on data models including material master information with all its associations:
Semantic Onboarding: Searching I_Product CDS View for importing Material Master semantics
Semantic Onbarding: Preview of Dependent Entities of I_Product object
Data Builder: Review of I_Product Table and Associations with other views and objects pt 1
Data Builder: Review of I_Product Table and Associations with other views and objects pt 2
Data Builder: Data Lineage of I_Product on current usage in the Data Builder
Learn more about Semantic Onboarding here.
What about other SAP Cloud solutions? How does Datasphere leverage the Business Context of their data?
Even though this blog is focused to S/4HANA data, it is important for you to know that SAP Datasphere also provides Business Content to other SAP Cloud solutions. The Business Content are packages that includes pre-configurations to jump start analytics projects. Pre-configurations such as tables, views, facts models and analytics models, including SAP Analytics Cloud stories (prebuilt dashboards), among other data objects. These packages come out-of-the-box and you can import the ones you want in your Datasphere tenant.
Some of the packages include business content for SAP S/4HANA, SAP Ariba SAP Concur, SAP Signavio, SAP Success Factors, also for different Industries. Find more information here.
Need to Push SAP S/4HANA Data Out to a Hyperscaler Data Lake / 3rd Party Data System?
This can be a common scenario when you manage a big percentage of non-SAP data in your organization but still require certain SAP data to integrate with it. In the ever-growing data landscape, organizations may opt for 3rd party data lake solutions such as MS Azure, AWS, Google Cloud for this purpose.
In order to support customers in this journey, especially when the vast majority of data they are managing is not coming from SAP systems; SAP has launched a capability called “Premium Outbound Integration” to allow near real-time SAP data replication to 3rd party data solutions by using SAP Datasphere’s Replication Flows.
SAP Datasphere’s Replication Flows
*Restrictions to delta capture are listed here.
There are 2 main outbound replication options:
It’s inbound: therefore, I’m not explaining it further in this blog.Outbound data movement: Very handy when you want to leverage Datasphere’s integration, modeling and data transformation capabilities on SAP data (and non-SAP data as well, depending on your use case), before replicating outputs to a 3rd party data system (like Azure as shown in the picture below)Pass-through option: Use Datasphere as a middleware to supported targets without any data transformation.
For more information about supported target systems, check the documentation here.
Important note: Datasphere provides the ability to replicate SAP data by leveraging ODP Data Replication APIs, leveraging Change Data Capture mechanisms from SAP systems such as S/4HANA, ECC and BW. SAP has an unpermitted usage of such ODP APIs for direct consumption in 3rd party systems, without an SAP solution such as Datasphere, 3rd party systems can consume SAP data through OData APIs, which may not be the best for consuming high volumes of SAP data. Learn more on this note.
Considerations of moving SAP data out
Although the Premium Outbound capability allows you to replicate SAP data out, there are some caveats you need to be aware when trying to preserve Business Context of the SAP data, being more specifically: from S/4HANA’s CDS views.
Business Context like Currency Conversions, Unit of Measures, Multi-Language, Hierarchies and Security Configurations can be lost when replicated out, therefore you may need to do an extra work in your target data system to rebuild that context that has been lost.
Currency Conversions:
SAP Datasphere respects the relationships of the CDS views and its associations. SAP Datasphere applies currency conversion on the fly for local currency, company currency, global currency, etc. maintaining the business context of SAP data intact.
However, when the need is to replicate this SAP data out to a 3rd party data system (such as any of the ones mentioned above), without this enrichment you would need to select one currency of choice to export OR create a custom model for each currency converted amount in the target system. This can be cumbersome as you would need to process a higher number of measures and key figures.
For instance, when you have a dynamic nature of currency conversions (like in Datasphere) you can receive the amount on the needed currency converted amount. On the other hand, you would need to design the model with multiple columns (CAD, EUR, USD, etc). To mention a real-life scenario: when managing pricing condition tables, you’ll need to multiply by the number of currencies you manage in your organization.
Unit of Measure Conversion
Similar to the Currency Conversions, SAP Datasphere has standard process to handle Unit of Measures (UoM) conversions, respecting the context coming from S/4HANA and making it easier to “translate” measures like (kg, gr, lbs, lt, gallons, etc.), especially when companies operates in multiple countries that have different measurement standards, requirements and specific company measures (like a palette of an specific product equals to 9 boxes in one country but in another equals to 12).
In the same way, when the need is to replicate this SAP data to a 3rd party data system, without the enrichment you would need to select one kind of UoM when exporting, and its business rules and context would need to be rebuilt in the 3rd party target system, potentially making it again a cumbersome process to deal with.
Multi-Language:
When leveraging SAP Datasphere for managing Analytics with SAP Data, the solution also provides the ability to pull up all the languages included in the source SAP system so that you can choose. It carries the texts of the master data in various languages. Take a look at this blog by @san_tran for an easy example.
On the other hand, when replicating the data out, you would need to select one language upfront (let’s say English) and the rest of the context (texts in other languages) would need to be rebuilt in your target 3rd party data system. For this last, you would need to extract the text and the data and then separately extract every text description of your master data (for example material master) and then rejoin them again in your target 3rd party data system. This could be also cumbersome, however the option is available for use cases when it makes sense to move SAP data out to another non-SAP data system.
Hierarchies:
As SAP Datasphere leverages the Semantic Onboarding, this makes it so much easier to import semantics and associations, respecting business context of the data including, of course, hierarchies.
Similar to the previous examples, when replicating data out to a 3rd party data system, there is no accelerator to assembly hierarchies in the target system, therefore an extra and very cumbersome work would be needed to rebuilt hierarchies. Let’s picture an automotive company scenario where you are dealing with millions of auto parts, multiple hierarchies including financial ones and versioning of those hierarchies. These could be expensive to maintain from scratch.
If your use case still needs to export SAP data out, take into consideration that you might need to flatten your hierarchies to become level in columns. Here users could only identify each level by their level name or number for drill down purposes (losing dynamic functionality as hierarchies might not be flexible anymore). If the wish is not to flatten hierarchies, then the rebuilt of those hierarchies would be needed to wiring them to your transaction data in the target system, this could represent higher costs as they can be multiple ones and maintenance of such hierarchies would be needed as well.
Security Configurations:
Data access controls are inherited from S/4HANA users into Datasphere. SAP BW/4HANA and SAP ECC with embedded BW authorizations can be imported into SAP Datasphere as well, making it easier to map and respect data access controls.
The approach leverages existing authorization objects from SAP S/4HANA to enforce row-level security in SAP Datasphere with Data Access Controls. Check this blog from @JensBraun to learn more about how to leverage S/4HANA authorizations into SAP Datasphere.
When replicating data out, all security configurations or data access controls get lost, leaving the unavoidable need to recreate security data access control in the 3rd party target data system.
Conclusions
Many organizations ask what the best solution is to manage SAP data for analytics purposes. SAP Datasphere is the spot on enterprise data warehouse solution in the cloud for this purpose. It respects and maintains the business associations of the SAP data, accelerating time to analytics with business context intact. Check my previous blog to revisit 10 reasons why SAP Datasphere makes sense for your data and analytics strategy beyond maintaining the business context of your data (like enabling a business data fabric architecture).
When managing SAP focused data use cases, the recommendation is to move forward with SAP Datasphere, leverage native connections and SAP business content. SAP Datasphere also provides multiple connectors to non-SAP sources, which means you can use Datasphere for combined enterprise-wide analytics.
Additional note: Recently, my team delivered a value case analysis based on the use of Datasphere to accelerate time to analytics while reusing and maintaining business context from S/4HANA and leveraging self-service tooling for data modeling in a Canadian Public Sector customer, resulting in a potential improvement in productivity of 50% in data modeling and in 1575 hours saved annually.
In addition, check this Gigaom field study on ROI on Datasphere, where they discovered that by using Datasphere there is an average of 72% accelerated data modeling and an approximate of 138% three-year TCO reduction vs a DIY implementation. Check the report here.
When does it make sense to replicate SAP data out? When you have other data use cases beyond SAP, or perhaps you have already invested in those non-SAP data cases with another technology, but you still want to use SAP-specific data and rebuilding the business context of the required SAP-specific data is not that cumbersome in terms of time and resource consumption in your organization.
I want to appreciate my data mentor Tony Cheung for his constant guidance which also made it possible for me to publish this blog.
Do you want to keep learning about SAP Datasphere?
SAP continues to add innovations into Datasphere and you can check them publicly in the roadmap.
SAP also continues to grow partnerships with category leaders such as Collibra, Databricks, Google Cloud, among others, to deliver even more effective, efficient and holistic enterprise data and analytics strategies for a wide range of organizations.
Also, take a look at these learning journeys on the SAP Learning Portal.
IntroductionYou probably have heard before that one of the strongest muscles that SAP Datasphere brings to the table is that it leverages, respects and maintains SAP business context from S/4HANA’s semantic layer.As Business Context I refer to: Metadata, Hierarchies, Currency Conversions, Unit of Measures, Multi-Language, Security Configurations; in other words, all objects associations that give business meaning to the data.As an example: let’s picture a simple yet common use case in supply chain analytics: you would like to understand the impact of discount negotiations you have had with your vendors/business partners in your procurement processes. Connections amongst your business partners (master data) and their related purchase orders, currency rates across countries (currency conversions), where they operate and in what levels, like international, national and regional geographic-wise (hierarchies) -just to mention few examples- act as the “glue” (business context) that gives business meaning on the “raw” data.In my previous blog, I talk about the great flexibility that SAP Datasphere offers for data integration to enable Business Data Fabric Architectures. Especially when leveraging data federation mechanisms to reduce storage costs and copies of the data. However, it is known that depending on the use case and performance on the usage of high volumes of data, data replication might be the best option.S/4HANA plays a key role in this journey, as the ERP Cloud core system, S/4HANA combines the power of transactions and analytical engine to expose enterprise data through virtual data models that are consumed with Core Data Service (CDS) views.Additional note: SAP S/4HANA provides over 8000 CDS views where around 20% are delta capable. These CDS views are out-of-the box available for Data Extraction, and the ones that are not, can be data-extraction enabled by creating custom CDS views which retrieve the respective associations and semantic relationships they have with other views and objects.Check this awesome blog from @FrankRiesner for more information about released data extraction enabled CDS views per S/4HANA version.Let’s see know why Datasphere is the spot on solution to maintain the business context of your SAP Data. Semantic Onboarding – Leverage your SAP Data in DatasphereWhether your SAP data consumption is through federation or replication, the Semantic Onboarding capability is an excellent way to provide visibility to technical and business data users of all released CDS views and their relationships/associations with other objects. The way that it works is that you actually import SAP semantics to Datasphere, enabling easy and earlier access to those views for data modeling and ultimately analytics consumption.SAP Datasphere provides mechanisms to maintain the semantics of the SAP data for many business context objects such as the ones I mentioned above: Metadata, Hierarchies, Currency Conversions, Unit of Measures, Multi-Language, Security Configurations.The Semantic Onboarding supports different source SAP systems, such as S/4HANA Cloud, S/4HANA, BW/4HANA, the BW Bridge and soon calculation views from HANA Cloud will also be supported, see the roadmap item.Let’s see a quick example of how the Semantic Onboarding looks and works. Here, I’m leveraging a connection to an S/4HANA Cloud system and importing the I_Product CDS View, to later see it available as a table in the Data Builder, where data analysts can start working on data models including material master information with all its associations: Semantic Onboarding: Searching I_Product CDS View for importing Material Master semanticsSemantic Onbarding: Preview of Dependent Entities of I_Product objectData Builder: Review of I_Product Table and Associations with other views and objects pt 1Data Builder: Review of I_Product Table and Associations with other views and objects pt 2Data Builder: Data Lineage of I_Product on current usage in the Data BuilderLearn more about Semantic Onboarding here. What about other SAP Cloud solutions? How does Datasphere leverage the Business Context of their data?Even though this blog is focused to S/4HANA data, it is important for you to know that SAP Datasphere also provides Business Content to other SAP Cloud solutions. The Business Content are packages that includes pre-configurations to jump start analytics projects. Pre-configurations such as tables, views, facts models and analytics models, including SAP Analytics Cloud stories (prebuilt dashboards), among other data objects. These packages come out-of-the-box and you can import the ones you want in your Datasphere tenant.Some of the packages include business content for SAP S/4HANA, SAP Ariba SAP Concur, SAP Signavio, SAP Success Factors, also for different Industries. Find more information here. Need to Push SAP S/4HANA Data Out to a Hyperscaler Data Lake / 3rd Party Data System?This can be a common scenario when you manage a big percentage of non-SAP data in your organization but still require certain SAP data to integrate with it. In the ever-growing data landscape, organizations may opt for 3rd party data lake solutions such as MS Azure, AWS, Google Cloud for this purpose.In order to support customers in this journey, especially when the vast majority of data they are managing is not coming from SAP systems; SAP has launched a capability called “Premium Outbound Integration” to allow near real-time SAP data replication to 3rd party data solutions by using SAP Datasphere’s Replication Flows.SAP Datasphere’s Replication Flows*Restrictions to delta capture are listed here.There are 2 main outbound replication options:It’s inbound: therefore, I’m not explaining it further in this blog.Outbound data movement: Very handy when you want to leverage Datasphere’s integration, modeling and data transformation capabilities on SAP data (and non-SAP data as well, depending on your use case), before replicating outputs to a 3rd party data system (like Azure as shown in the picture below)Pass-through option: Use Datasphere as a middleware to supported targets without any data transformation.For more information about supported target systems, check the documentation here.Important note: Datasphere provides the ability to replicate SAP data by leveraging ODP Data Replication APIs, leveraging Change Data Capture mechanisms from SAP systems such as S/4HANA, ECC and BW. SAP has an unpermitted usage of such ODP APIs for direct consumption in 3rd party systems, without an SAP solution such as Datasphere, 3rd party systems can consume SAP data through OData APIs, which may not be the best for consuming high volumes of SAP data. Learn more on this note. Considerations of moving SAP data outAlthough the Premium Outbound capability allows you to replicate SAP data out, there are some caveats you need to be aware when trying to preserve Business Context of the SAP data, being more specifically: from S/4HANA’s CDS views.Business Context like Currency Conversions, Unit of Measures, Multi-Language, Hierarchies and Security Configurations can be lost when replicated out, therefore you may need to do an extra work in your target data system to rebuild that context that has been lost. Currency Conversions:SAP Datasphere respects the relationships of the CDS views and its associations. SAP Datasphere applies currency conversion on the fly for local currency, company currency, global currency, etc. maintaining the business context of SAP data intact.However, when the need is to replicate this SAP data out to a 3rd party data system (such as any of the ones mentioned above), without this enrichment you would need to select one currency of choice to export OR create a custom model for each currency converted amount in the target system. This can be cumbersome as you would need to process a higher number of measures and key figures.For instance, when you have a dynamic nature of currency conversions (like in Datasphere) you can receive the amount on the needed currency converted amount. On the other hand, you would need to design the model with multiple columns (CAD, EUR, USD, etc). To mention a real-life scenario: when managing pricing condition tables, you’ll need to multiply by the number of currencies you manage in your organization. Unit of Measure ConversionSimilar to the Currency Conversions, SAP Datasphere has standard process to handle Unit of Measures (UoM) conversions, respecting the context coming from S/4HANA and making it easier to “translate” measures like (kg, gr, lbs, lt, gallons, etc.), especially when companies operates in multiple countries that have different measurement standards, requirements and specific company measures (like a palette of an specific product equals to 9 boxes in one country but in another equals to 12).In the same way, when the need is to replicate this SAP data to a 3rd party data system, without the enrichment you would need to select one kind of UoM when exporting, and its business rules and context would need to be rebuilt in the 3rd party target system, potentially making it again a cumbersome process to deal with. Multi-Language:When leveraging SAP Datasphere for managing Analytics with SAP Data, the solution also provides the ability to pull up all the languages included in the source SAP system so that you can choose. It carries the texts of the master data in various languages. Take a look at this blog by @san_tran for an easy example.On the other hand, when replicating the data out, you would need to select one language upfront (let’s say English) and the rest of the context (texts in other languages) would need to be rebuilt in your target 3rd party data system. For this last, you would need to extract the text and the data and then separately extract every text description of your master data (for example material master) and then rejoin them again in your target 3rd party data system. This could be also cumbersome, however the option is available for use cases when it makes sense to move SAP data out to another non-SAP data system. Hierarchies:As SAP Datasphere leverages the Semantic Onboarding, this makes it so much easier to import semantics and associations, respecting business context of the data including, of course, hierarchies.Similar to the previous examples, when replicating data out to a 3rd party data system, there is no accelerator to assembly hierarchies in the target system, therefore an extra and very cumbersome work would be needed to rebuilt hierarchies. Let’s picture an automotive company scenario where you are dealing with millions of auto parts, multiple hierarchies including financial ones and versioning of those hierarchies. These could be expensive to maintain from scratch.If your use case still needs to export SAP data out, take into consideration that you might need to flatten your hierarchies to become level in columns. Here users could only identify each level by their level name or number for drill down purposes (losing dynamic functionality as hierarchies might not be flexible anymore). If the wish is not to flatten hierarchies, then the rebuilt of those hierarchies would be needed to wiring them to your transaction data in the target system, this could represent higher costs as they can be multiple ones and maintenance of such hierarchies would be needed as well. Security Configurations:Data access controls are inherited from S/4HANA users into Datasphere. SAP BW/4HANA and SAP ECC with embedded BW authorizations can be imported into SAP Datasphere as well, making it easier to map and respect data access controls.The approach leverages existing authorization objects from SAP S/4HANA to enforce row-level security in SAP Datasphere with Data Access Controls. Check this blog from @JensBraun to learn more about how to leverage S/4HANA authorizations into SAP Datasphere.When replicating data out, all security configurations or data access controls get lost, leaving the unavoidable need to recreate security data access control in the 3rd party target data system. ConclusionsMany organizations ask what the best solution is to manage SAP data for analytics purposes. SAP Datasphere is the spot on enterprise data warehouse solution in the cloud for this purpose. It respects and maintains the business associations of the SAP data, accelerating time to analytics with business context intact. Check my previous blog to revisit 10 reasons why SAP Datasphere makes sense for your data and analytics strategy beyond maintaining the business context of your data (like enabling a business data fabric architecture).When managing SAP focused data use cases, the recommendation is to move forward with SAP Datasphere, leverage native connections and SAP business content. SAP Datasphere also provides multiple connectors to non-SAP sources, which means you can use Datasphere for combined enterprise-wide analytics.Additional note: Recently, my team delivered a value case analysis based on the use of Datasphere to accelerate time to analytics while reusing and maintaining business context from S/4HANA and leveraging self-service tooling for data modeling in a Canadian Public Sector customer, resulting in a potential improvement in productivity of 50% in data modeling and in 1575 hours saved annually.In addition, check this Gigaom field study on ROI on Datasphere, where they discovered that by using Datasphere there is an average of 72% accelerated data modeling and an approximate of 138% three-year TCO reduction vs a DIY implementation. Check the report here.When does it make sense to replicate SAP data out? When you have other data use cases beyond SAP, or perhaps you have already invested in those non-SAP data cases with another technology, but you still want to use SAP-specific data and rebuilding the business context of the required SAP-specific data is not that cumbersome in terms of time and resource consumption in your organization.I want to appreciate my data mentor Tony Cheung for his constant guidance which also made it possible for me to publish this blog. Do you want to keep learning about SAP Datasphere? SAP continues to add innovations into Datasphere and you can check them publicly in the roadmap.SAP also continues to grow partnerships with category leaders such as Collibra, Databricks, Google Cloud, among others, to deliver even more effective, efficient and holistic enterprise data and analytics strategies for a wide range of organizations. Also, take a look at these learning journeys on the SAP Learning Portal. Read More Technology Blogs by SAP articles
#SAP
#SAPTechnologyblog
