EMA Blog Community

Social media’s potential for improving IT management has been a hot topic recently. ServiceNow continues to emphasize the collaborative nature of their platform, and CA is also bringing forth new solutions here with their OpenSpace technology. Improving teamwork, leveraging community expertise, and empowering the end user are all important objectives and (as I’ll note in a forthcoming whitepaper for CA) the service desk must become social or become irrelevant.

On another front, Big Data techniques are finding their way into IT service management as well. Vendors like Prelert are applying “self-learning analytics” to identify repeatable problem signatures, and succeeding in selling this to some of the largest IT organizations.

The convergence of these two trends is already in progress. What is a social media stream but just another event feed? What does it matter that the “endpoints” are people rather than devices? Patterns are patterns, and I believe that signatures of considerable interest lurk in the firehoses of social interaction surrounding IT service delivery.

Prelert is showing us how to do this with systems management data (automated event streams). The human-generated event streams are different in that we don’t understand their semantics quite as readily. Enter text analytics. Basic ground floor capabilities include things like understanding word stems (certification and certify both have a stem of certif), synonyms, and multiple word terms. Beyond that, statistical techniques such as clustering can be applied to understand emergent patterns.

There’s a lot of low hanging fruit in text analytics for IT management. The first priority would be incident management. Last year, I came across this SAS paper on incident management in the oil and gas industries — so-called “crater” industries, because that’s what incidents lead to. (The white paper is #9961 and titled “Text Mining for Safety,” if the deep link doesn’t work for you) This paper was an eye opener and I consider it required reading for all IT incident managers and vendors of IT service desk tools.

Quote: “Structured data collected by check boxes or “yes” or “no” fields make it easy for analysis and reporting purposes. But filling out lengthy questionnaires on so many structured data fields during the actual data input makes the data entry process very cumbersome.” Especially when, you know, an OIL RIG IS EXPLODING or little things like that. Bottom line, incident reporting is best captured as narrative – people as close as possible to the action spilling their guts via typing or even speech to text capture without a lot of monkeying with a complex workflow UI.

Then, with the advancement of text mining techniques and suitably powerful computers, SAS makes the very sensible point that the categorization and analysis can be done after the fact. Let the computer do the work. Look for patterns in multiple incidents, common themes, consistency across reports, and emerging trends.

SAS and SPSS have the generic capabilities, but, speaking from experience, they are not trivial to apply to a given problem domain. The market is therefore wide open for IT management vendors to start baking in predefined, simplified text analytics into all kinds of ITSM tools and processes.

I’ve had a couple of conversations lately with SoftLib, who as far as I (and they) know are the first vendor to market with a text analytic offering specifically marketed for IT service management. They have a very interesting story on improving incident management and time to resolution through text analytics, and understand the ITSM world well.

Verint has also started down this road, but more from a general call center perspective, not as specific to IT management.

On a different topic, another compelling text analytics story that’s been out for a few years is how SAS was used for the HP/Compaq merger. Think about the product line challenge in such a large merger – and how you would in any reasonable time frame combine them.

Again, enter text analytics. The team performing the merger took all the relevant documentation from both companies’ product lines, had a summer intern run it through the analytics engine, and developed a taxonomy that proved 85% accurate. (TDWI writeup here.)

Why is this relevant to IT management? IT portfolio management. Human beings behave entrepreneurially, and this leads to redundancy, as enterprise architects well know. How to rationalize a portfolio of 5,000 applications is a nontrivial question. Manual analysis and categorization doesn’t scale; either it takes too long or it is too inconsistent. On the other hand, if you have documentation and narrative – the more the better – characterizing each application in the portfolio, again, the computer can do the work. A half an hour interview on each application’s major business processes, data subjects, and technical architecture would be more than enough material to apply text mining to the thorny problem of app redundancy. Similarly for vendor products, which (the efforts of analyst firms notwithstanding) also tend to proliferate redundantly in large enterprises.

I don’t know anyone marketing text mining for IT portfolio management, but it makes too much sense for someone not to. Drop me a line when you do.

Finally, why the title of this blog? I haven’t talked about IBM’s Watson up to this point, but text analytics is a big part of how it works. As I think about the day to day work of IT, I envision a future – not too distant – where Watson, or some descendant, is there for me on call, as a part of the contextual intelligence characterizing the post-PC world. Not only to answer my questions, but to volunteer interesting information I may have overlooked – emergent patterns, trends, and possibly causal relationships, rank ordered as to significance. This agent will be constantly monitoring incident logs, system documentation, social media, even (where appropriate) verbal human conversations. (Big strides were recently announced in Scientific American in solving the “cocktail party problem” of understanding multiple overlapping human voices.)

It’s no secret that IBM has identified call centers as a logical application of Watson. Of course, such applications at first will be targeted to the largest, most strategic uses, where IBM will make plenty of money. I’m wondering about the whole ecosystem around text analytics and natural language processing for IT management … across large, medium, and small accounts and channels. IBM won’t own all of it, and we don’t need all of Watson to start realizing significant value now. Other innovators are already moving in.

I’m thinking of the Starship Enterprise computer and how it was called on to assist the bridge crew. What if that kind of intelligence was dialed into your incident management conference bridges? Or there to be consulted on proposals for new systems or vendor products? The first stirrings of this kind of assistance are already visible.

Related articles

• IBM Watson and Medical Records Text Analytics (medicineandtechnology.com)