The CMDB is not a data warehouse
Managing IT data is maddening. IT organizations are hamstrung right and left by lack of complete and high quality information, yet approaches to resolve this have been only partially successful – at best!
One approach that has had less attention is the idea of a data warehouse for IT data. As big data and analytics get more and more traction, the idea of applying them to IT is obvious.
Now, a CIO, if presented with the idea of an “IT data warehouse,” may respond “I thought we had one of those – the CMDB (Configuration Management Database, or System).” But the CMDB/CMS does not fit the classic data warehouse definition in important respects, especially in that data warehouses are typically read-only (non-volatile).
In reality, the CMDB has become more of a transactional back-end for production operations. The “CMDB” found at the heart of modern IT service management systems (such as BMC Atrium) may have extensive integrations, but this is not sufficient to consider it a data warehouse.
Often, a CMDB accepts direct updates of its data. The needed reporting “dimensions” may not exist in the CMDB. Although it may have audit trails, it typically is not rigorously historical in the sense of a true data warehouse. If the CMDB is running on the same system supporting high volume Incident and Service Desk processes, long running analytic queries might seriously hamper performance.
All in all, the CMDB is a poor analytic platform.
In data warehousing terms, the CMDB is more akin to what Bill Inmon termed the “Operational Data Store” or ODS . The ODS, while it integrates data, is current state oriented and can be transactional. An ODS may feed an enterprise data warehouse, but should never be confused with one. Similarly, the CMDB may feed the IT data warehouse, but should not be confused with it. (In fact, a CMDB is not required to have an IT data warehouse.)
An IT data warehouse can be assembled with off the shelf tools, but this is labor intensive and vendor offerings can add substantial value. A number of players are offering solutions that are not CMDBs. Rather, these products draw from the CMDB/CMS as one of various sources:
- HP continues to invest in the warehouse capability underlying its IT Executive Scorecard and IT Financial Management solutions (EMA white paper).
- SAS also continues to evolve its IT Resource Management solution, which seeks to provide value for both resource/capacity and IT financial management.
- Blazent solves difficult problems with IT asset and inventory reconciliation via integrating and analyzing a multitude of sources.
- PureShare offers its SingleView IT operational IT performance reporting, integrating sources such as service desk and IT financial management
- Apptio has a powerful data profiling and reconciliation architecture it employs in its Technology Business Management solution.
- Emerging player Northcraft Analytics has offerings for both Remedy and ServiceNow metrics and KPIs.
These solutions are more at the “business of IT” level. At the element management and in the governance, risk and compliance domains we see many more interesting and specialized data aggregators and analytics engines, such as Splunk, Arcsight, Rev2, Prelert, and many more. See also Watson joins the IT management team.
As data warehousing is an extensible and scalable approach, many of these vendors are well positioned to move from their origins into more cross functional problems. An integrated data architecture has powerful network effects, as each new integrated source enables an exponentially increasing number of analysis use cases.
What’s the downside? Services, in a word. These are not, and never will be, plug and play solutions. Sourcing, extracting, transforming, and loading the data takes investment, especially from lesser known or custom built sources. You may spend as much on development as you do on acquisition. You might be tempted to do it all from scratch, but be careful there; sound data warehouse architectures take a lot of thought. (How will you handle slowly changing dimensions?)
Furthermore, the very term “data warehouse” may have negative connotations in some organizations, as an overly heavyweight, expensive proposition. Proposing a data warehouse on the heels of a failed CMDB implementation will probably not win you any friends. But data warehousing remains a vital and growing practice area in the overall IT landscape. When carefully considered and executed, it’s led to many successes. Just go to a TDWI course or two.
As always, thoughts appreciated.
Posted in Uncategorized Tags: Bill Inmon, CMDB, CMS, Configuration Management Database, Configuration Management System, Datawarehouse, IT Data Warehouse, IT Management, ODS, Operational data store, Performance indicator, Service Desk, ServiceNow
6 Responses to The CMDB is not a data warehouse
Leave a Reply
-
Subscribe
Subscribe to Integrated IT Management
New Research- From Systems to Service Management: Running Your IT Department Like a Business May 30, 2012 Charlie Betz
- Real-World CMDB May 24, 2012 Charlie Betz
- Identifying and Tracking Applications: A Detailed Look May 7, 2012 Charlie Betz
- The Trusted CMDB: Data Quality and Governance May 1, 2012 Charlie Betz
- Turbocharging Your CMDB April 25, 2012 Charlie Betz
-
More EMA Bloggers
oh now you’ve done it. Another drum for the vendors to bang
At the big end of town where you operate Charles I can see the value in this, just like CMDB. And just like CMDB it shouldn’t be attempted by the other 95%
Although dwh has more chance of being automated and delivered OOTB than CMDB does which would make it accessible to more. I thought all of the suites (HP, CA, IBM, BMC) already had this for performance and capacity analysis
RE: The Skeptic
Hi Skeptic,
Hope all is well in the land of fear, uncertainty and doubt! It’s true that in the big end of town this type of functionality is more heavily desired. I’m sure you’ve laid out in other places why you don’t think a CMDB is necessary, but… I’ll lay out some reasons as to why I think CMDB should be implemented by organizations with 100 IT Staff and above (95% is a little too broad in my opinion):
- Incident History by CI – By seeing the history of Incidents on a CI (especially in aggregate by Manufacturer/Model), you can gather your own data to hand over to the procurement department to beat up vendors on price. Seems good to me.
- History of Changes on CI – By seeing the history of changes on a CI (including current changes), you can help prevent unplanned outages.
- Establish accountability quickly for IT equipment incidents, changes, problems, releases and requests – Sure, there may be a director of windows servers, but who owns hostname blahblah@fairlylargeorg.com? I need to know immediately for project planning/restoration/performance issues/etc…
Are you skeptical because so many CMDB initiatives fail? Is it the practicality of the implementation that causes your skepticism? The major problems I’ve seen arise in CMDB are around establishing accountability and responsibility, not the technical aspects. For example, I’ve seen breakdowns when trying to get the proper security credentials to go handle the discovery phase. That requires some sponsorship! But I just don’t think a CMDB implementation is that hard technically. I can see arguments from different angles around the people and process aspects though.
Anyway, on to the topic of your response about the big suites. I’ve been on a great number of large implementations for 3 of the 4 ones mentioned. So, I thought I’d address your comment. The answer is… they do have tools that help for performance and capacity analysis, but they are typically not integrated into a BI platform making the tools proprietary and not helpful to the executive team that needs the information. When a large vendor has had the foresight to integrate into the BI platform, they have run into the same challenges that my company is trying to solve (that Charles mentioned in the article)… Data Quality, ETL, the value of the metrics themselves, which process areas you focus on, etc.. And that ‘taint easy friend! As an example, Northcraft has only solved all of the world’s problems for 8 process areas and 2 platforms.
Lee
The data warehouse paradigm is more scalable (down as well as up) but has its own set of limitations. Because of the heavy services angle, I don’t expect to see a gold rush of vendors, but may be proven wrong.
The suite vendors have various aggregation points, but I’m looking for products that are on the agnostic side when it comes to sources. The list is by no means complete, I am sure. Work in progress…
Pingback: Northcraft Analytics and the IT Data Warehouse | Northcraft Analytics LLC
Dear Charles,
You have made an important point, showing the difficulties of CMDB, especially when it comes to take tactical and strategic IT management decisions. I think it is the same situation as we can find in finance: No one would decide on your investment strategy based on your accounting system, right?
On the other hand I do not think that a data warehouse will be a silver bullet for IT management. First, organizations will spend too much time to define and agree on their data model. From the business perspective this can easily be interpreted as IT is going back into the ivory tower to define their operating model.
Secondly, a data warehouse in most cases will only be beneficial, if you can feed in data from source systems. There are many organizations out there, not having their entire IT landscape covered with operational systems and low level data on it, but still they want to base their decisions on facts and analysis.
In my opinion it is more efficient to focus on management decisions, and therefore only use and collect data really needed. This could be guaranteed through a tool that is not only focusing on structure but already takes the management perspective.
Best,
André Christ
Hi Andre, sorry for approval delay. We’ve got some issue that the blogs aren’t forwarding the comments to email, which I rely on.
I agree with much of what you say. It’s always a challenge for those of us who specialize in understanding these systems to have to continually remind our readers that these systems come AFTER you define an operating model. Absolutely, there is no silver bullet, and any BI/DW strategy should start with the business problem being solved.
That said, at scale in maturing IT environments, DW/BI offers a proven set of practices for solving these problems, and a professional community very aware of the issues you are raising. See http://www.dama.org and http://www.tdwi.com. We’re also seeing vendors coming to market with predefined “BI for IT” offerings that can help with some of the effort traditionally associated with greenfield DW design & build.
Cheers,
Charlie