Data Warehouse "Optional"?
Date: 21/10/2013

BY DAVID MARTIN, PRINCIPAL CONSULTANT

We're now in a time when memory is inexpensive and advances in technology allows data to be held in memory for lightening performance, providing much faster insights into business data regardless of the grain, ie products such as SAP HANA and IBM TM1.

This type of information delivery relies heavily on the ability to integrate data rapidly from a variety of sources often supplied at different frequencies. For customers possessing the funds to procure the hardware and software to make this a reality, this just might deliver the quality and consistency that you need. But for the lion share of the market, those with both large quantities of information but limited funds with which to mobilise the infrastructure, you are left with an ""untidy kitchen"", where all of the preparation is done but never kept clean, entities rarely conformed and quality ignored or information dropped. For example SAP HANA is perfect if all of your business data is intregrated within SAP, but in reality this is not the case, and it's only part of the puzzle.

More companies are doing away with the development of a data warehouse all together and focusing on delivering information quickly.

To those of you nodding your heads let me put this to you. A data warehouse isn't something you need to shy away from and it's something that doesn't have to be 100% complete when you deliver to your business community. It's something that should be delivered and enhanced incrementally, just like Kimball has suggested for decades. The data warehouse should be seen as a way to hold important information, a gateway or single point to data, structured for easy access by a multitude of applications and tools ie., not just for reporting. Think of it more as a critical part to your organisations ""data flow"", like a hub and operating in both batch and real-time modes.

Take product or customers as an example. Both of these are crucial entities and often defined in several places around your business. One of the functions of your IM strategy is to ensure data is consistent across your systems. Why not use the place that retrieves, cleans and organises this information for business presentation, as the driver to ensure that CRM is up-to-date, using the content from the product master system. Perhaps using your customer details, so well defined in your practice management system, to be fed out to your CRM where users always have the correct contact information and history, as opposed to simply what the previous person noted down. This is to say the actuals.

We're also sometimes guilty of not looking at the big picture, at what the future entails. When you embark on your reporting strategy it won't take much additional effort to plan for data mining and predictive analytics. Ignoing sensible storage and tracking of key assets will only leave gaping holes when you come to adopt these more advanced methods of business analytics. A data warehouse doesn't mean one huge database, it can encompass different technologies to hold both your structured and unstructured data.

Your IM solution should use a data warehouse for what it represents, a single place to assess, report and deliver information - not only to the executives of the business, but to those other key business functions to deliver reduced complexity.

If you'd like any information on data warehousing, click here to send me an email.