Good question. As I am a consultant, my answer is – as always – “it depends” 😉. No booing, please. I will expand.
History always helps in explaining complex problems, so let us first address the question: “why do data warehouses even exist?”. Once upon a time, operational reports on data from a transactional system were created in the transactional system itself. As the underlying database of such a system was not very good in writing to and reading from database tables simultaneously, query processing for operational reports could impose such a burden on the database, that transaction processing – the main function of a transactional system – was hampered or even stopped. The well-known phenomenon of “a runaway query pulling down a database”. The solution then was the introduction of a separate system for query processing: a “data warehouse”. This separated “OnLine Analytical Processing” (OLAP) from “OnLine Transactional Processing” (OLTP). SAP introduced its own data warehouse system in 1998: SAP Business Warehouse or BW. And recently, SAP introduced a public cloud data warehouse named SAP Datasphere [https://community.sap.com/t5/technology-blogs-by-members/sap-analytics-landscape-of-the-future/ba-p/13863617 ].
But now you have the S/4HANA ERP software on top of HANA hardware, which to a great deal solves the problem with writing to and reading from database tables simultaneously. The “Embedded Analytics” functionality provides you with excellent options to build operational reports in the S/4HANA transactional system itself. So, do you still need that data warehouse? As I am a BW veteran with over 20 years of BW experience, at first I hoped the answer would be “yes”. But these days, as I earn my living more and more with S/4HANA Embedded Analytics, I couldn’t care less 😉. And to be honest, the answer often is “no”.What I currently do is visit clients that are struggling with this question armed with the checklist below of remaining use cases for a data warehouse:
historical data;data snapshots;cross-system data integration;extremely high data volumes.
Use case #1: historical data. “Old” data can be archived (although these days this rarely happens). It can also be left behind when moving to a new transactional system. The latter will quite often be the case when clients move from SAP ECC to S/4HANA. If this “historical data” is valuable for reporting, it needs to be stored elsewhere. S/4HANA Embedded Analytics can only work with data actively available in the S/4HANA database, so S/4HANA itself is not an option for reporting on historical data. Solution is to store the historical data in BW, Datasphere or a non-SAP data warehouse, combine it there with current data from S/4HANA, and report across historical and current data with a frontend tool – e.g. SAP Analytics Cloud – connected to this data warehouse. Datasphere provides the best options to combine “persistent” historical data with “virtual” current data.
Use case #2: data snapshots. Reporting users are aware that it is quite difficult to run a report one year later and obtain identical results as the year before, the reason lying in the fact that the environment, e.g. master data, has changed over the year. I was recently asked the question by one of my embedded analytics clients if it is possible to see old versions of certain conversion factors after they are changed in the database table. The answer is “no”. Real-time data is real-time data. If it is not stored anywhere in the database, it is gone and cannot be used. (Yes, I know, there are exceptions, time-dependent master data, change documents, but cut me some slack.) In cases in which this “no” is not acceptable, “data snapshots” can be applied. Here a dataset is stored including all “environmental parameters” in such way that a report on the dataset can be identically reproduced at a later time.
Use case #3: cross-system data integration. Probably the most important of the four remaining use cases for data warehousing. Question should not be if a company has more than one source system, because that is nearly always the case. And each system usually offers ways to report on the data within that system. Question therefore should be: is there a need to report across two or more of these systems? The best example in my career was a valuable report on SAP ERP data combined with Microsoft CRM data with data from both systems being strongly intertwined. Here SAP BW did an excellent job in fulfilling this requirement. The cloud landscape picture of my previous blog shows graphically how Datasphere can integrate data from S/4HANA, other SAP cloud systems and non-SAP systems.
Use case #4: extremely high data volumes. Here one should not think of millions of records – a HANA database cuts through millions of records like butter – but more like billions of records. In general, you will only encounter this use case in a retail environment. Example: all lines of a shopping receipt entering a transaction system, while one is not interested in the fact that Sat September 14th at 09:16 AM Freek Keijzer bought 4 bags of Douwe Egbert Roodmerk coffee with “hamster discount”. In this example it makes sense to start with aggregated data in a data warehouse.
And what tends to be the result of my visits to clients armed with the four-point checklist? To be honest, I have not encountered data snapshots or extremely high data volumes yet, but that can be due to our customer base. But the use cases historical data and cross-system data-integration I do encounter. In most cases the data warehouse can be noteworthy reduced and in some cases even phased out, after an effective implementation of embedded analytics. If the embedded analytics itself does not bring joy to the client, then the TCO reduction it causes will.
While we are on the topic of TCO reduction, what about in-memory reporting tools like QlikView and EveryAngle? These tools are well adopted in the SAP world as they were earlier with in-memory technology than SAP itself, but this advantage has disappeared, so now it is time to replace them by embedded analytics. Did I already mention that embedded analytics comes for free with the S/4HANA licenses? A no-brainer, really.
Closing remarks, always difficult:
SAP S/4HANA comes with Embedded Analytics, which is cool and free of (additional) costs.If you have S/4HANA with embedded analytics and you also have a data warehouse, your data warehouse can become smaller. In some cases even so small it … disappears.Remaining use cases for a data warehouse are: historical data, data snapshots, cross-system data-integration and extremely large data volumes.If you have in-memory reporting tools like QlikView and EveryAngle, you should get rid of them.
Again, feedback is much appreciated.
Good question. As I am a consultant, my answer is – as always – “it depends” 😉. No booing, please. I will expand.History always helps in explaining complex problems, so let us first address the question: “why do data warehouses even exist?”. Once upon a time, operational reports on data from a transactional system were created in the transactional system itself. As the underlying database of such a system was not very good in writing to and reading from database tables simultaneously, query processing for operational reports could impose such a burden on the database, that transaction processing – the main function of a transactional system – was hampered or even stopped. The well-known phenomenon of “a runaway query pulling down a database”. The solution then was the introduction of a separate system for query processing: a “data warehouse”. This separated “OnLine Analytical Processing” (OLAP) from “OnLine Transactional Processing” (OLTP). SAP introduced its own data warehouse system in 1998: SAP Business Warehouse or BW. And recently, SAP introduced a public cloud data warehouse named SAP Datasphere [https://community.sap.com/t5/technology-blogs-by-members/sap-analytics-landscape-of-the-future/ba-p/13863617 ].But now you have the S/4HANA ERP software on top of HANA hardware, which to a great deal solves the problem with writing to and reading from database tables simultaneously. The “Embedded Analytics” functionality provides you with excellent options to build operational reports in the S/4HANA transactional system itself. So, do you still need that data warehouse? As I am a BW veteran with over 20 years of BW experience, at first I hoped the answer would be “yes”. But these days, as I earn my living more and more with S/4HANA Embedded Analytics, I couldn’t care less 😉. And to be honest, the answer often is “no”.What I currently do is visit clients that are struggling with this question armed with the checklist below of remaining use cases for a data warehouse:historical data;data snapshots;cross-system data integration;extremely high data volumes.Use case #1: historical data. “Old” data can be archived (although these days this rarely happens). It can also be left behind when moving to a new transactional system. The latter will quite often be the case when clients move from SAP ECC to S/4HANA. If this “historical data” is valuable for reporting, it needs to be stored elsewhere. S/4HANA Embedded Analytics can only work with data actively available in the S/4HANA database, so S/4HANA itself is not an option for reporting on historical data. Solution is to store the historical data in BW, Datasphere or a non-SAP data warehouse, combine it there with current data from S/4HANA, and report across historical and current data with a frontend tool – e.g. SAP Analytics Cloud – connected to this data warehouse. Datasphere provides the best options to combine “persistent” historical data with “virtual” current data.Use case #2: data snapshots. Reporting users are aware that it is quite difficult to run a report one year later and obtain identical results as the year before, the reason lying in the fact that the environment, e.g. master data, has changed over the year. I was recently asked the question by one of my embedded analytics clients if it is possible to see old versions of certain conversion factors after they are changed in the database table. The answer is “no”. Real-time data is real-time data. If it is not stored anywhere in the database, it is gone and cannot be used. (Yes, I know, there are exceptions, time-dependent master data, change documents, but cut me some slack.) In cases in which this “no” is not acceptable, “data snapshots” can be applied. Here a dataset is stored including all “environmental parameters” in such way that a report on the dataset can be identically reproduced at a later time.Use case #3: cross-system data integration. Probably the most important of the four remaining use cases for data warehousing. Question should not be if a company has more than one source system, because that is nearly always the case. And each system usually offers ways to report on the data within that system. Question therefore should be: is there a need to report across two or more of these systems? The best example in my career was a valuable report on SAP ERP data combined with Microsoft CRM data with data from both systems being strongly intertwined. Here SAP BW did an excellent job in fulfilling this requirement. The cloud landscape picture of my previous blog shows graphically how Datasphere can integrate data from S/4HANA, other SAP cloud systems and non-SAP systems.Use case #4: extremely high data volumes. Here one should not think of millions of records – a HANA database cuts through millions of records like butter – but more like billions of records. In general, you will only encounter this use case in a retail environment. Example: all lines of a shopping receipt entering a transaction system, while one is not interested in the fact that Sat September 14th at 09:16 AM Freek Keijzer bought 4 bags of Douwe Egbert Roodmerk coffee with “hamster discount”. In this example it makes sense to start with aggregated data in a data warehouse.And what tends to be the result of my visits to clients armed with the four-point checklist? To be honest, I have not encountered data snapshots or extremely high data volumes yet, but that can be due to our customer base. But the use cases historical data and cross-system data-integration I do encounter. In most cases the data warehouse can be noteworthy reduced and in some cases even phased out, after an effective implementation of embedded analytics. If the embedded analytics itself does not bring joy to the client, then the TCO reduction it causes will.While we are on the topic of TCO reduction, what about in-memory reporting tools like QlikView and EveryAngle? These tools are well adopted in the SAP world as they were earlier with in-memory technology than SAP itself, but this advantage has disappeared, so now it is time to replace them by embedded analytics. Did I already mention that embedded analytics comes for free with the S/4HANA licenses? A no-brainer, really.Closing remarks, always difficult:SAP S/4HANA comes with Embedded Analytics, which is cool and free of (additional) costs.If you have S/4HANA with embedded analytics and you also have a data warehouse, your data warehouse can become smaller. In some cases even so small it … disappears.Remaining use cases for a data warehouse are: historical data, data snapshots, cross-system data-integration and extremely large data volumes.If you have in-memory reporting tools like QlikView and EveryAngle, you should get rid of them.Again, feedback is much appreciated. Read More Technology Blogs by Members articles
#SAP
#SAPTechnologyblog
