Last week, I FINALLY published the 2011 ROI Case Studies research, which I had planned to publish back in 2010. The delay was due to competing priorities on my side, and I’d like to thank Robert Zoch (Brainware), Sam Benihya (AppLoader), and Joe Fitzpatrick (Splunk) for hanging in there with me during the lengthy interview, review, and edit process.

The paper is entitled “Looking for ROI in all the Right Places: Return on Investment, 2011”, and is available for download at: We ended up with three great studies illustrating the hard ROI of the products. This blog is a “Cliff Notes” version.

AppLoader is a load testing solution for Citrix, PeopleSoft, Windows, client-server, Oracle, Siebel, SAP, and custom applications. The case study was about Corinthian Colleges, Inc. This is one of the largest for-profit, post-secondary education companies in North America, with more than 113,800 students at over 122 U.S. and Canadian campuses. Corinthian Colleges was in the process of upgrading all of the colleges to a new third party student management system.

The problem is that each college has a unique portal and is also unique in its personnel, classes, terms, schedules, etc., so each one had to be tested separately as it came online. Personnel from the specific college and department being upgraded tested the proposed upgrade by manually inputting a variety of use cases to make sure the change worked as expected. However, the process took weeks to complete, and testers weren’t pleased that they had to keep up with their “real” work while testing the new software. Because they were multi-tasking, testers often missed critical issues.

With AppLoader they automated the testing process, which made internal customers more  productive. AppLoader yielded ROI of over $1.5 million, based on staff time savings plus savings over licensing costs for a competing solution.

Brainware’s Distiller is an “intelligent data capture solution” which speeds processing of invoices, remittance documents, customer orders, and similar documents via a combination of Optical Character Recognition (OCR) and self-learning. The company profiled is an international mining company struggling to minimize the manual labor necessary for timely invoice entry. Prompt invoice payment was important because it enabled the company to take advantage of “early payment discounts” offered by suppliers.

Each of the company’s suppliers creates invoices in a different format. During deployment, Distiller is “shown” a generic set of invoices (usually about 30). The product is then able to recognize what new invoices look like regardless of layout or vendor, and to correctly identify the critical pieces of information on each invoice. From that small sample set, the program generalizes to understand hundreds of thousands of different invoices, since virtually every company’s invoices are unique. Some of Brainware’s customers have half a million vendors in their database, each with distinctive invoice formats. Using Brainware, the mining company chalked up ROI of $1.9 million, based on staff savings as well as capture of early payment discounts. In the process, the productivity of Accounts Payable clerks improved by 300%.

Last but not least is Splunk. I will be doing a webinar for Splunk, by the way, on Thursday, April 14 at 9 a.m. Pacific. Please feel free to attend. The registration link for the event is:

Splunk delivers “Operational Intelligence”, insight into IT infrastructure data (and business data as well) from multiple diverse sources. Users use Splunk to index, search, and report across metrics, logs, and any other type of structured or unstructured data to troubleshoot issues, find root causes, and investigate security incidents, for example. Splunk also enables comparison of real-time data against historical trend/baseline/KPI data.

The IT professional interviewed leads a team supporting the tiered applications of a large financial services company. Running a significant number of distributed, message-based applications, they found themselves engulfed in “millions of events per day”. Event messages came from hundreds of sources, including custom and third-party software, servers, c language code, Java code, Operating Systems, databases and others. “When something broke, we had too much information and no way to narrow it down. When we had problems with our web application servers—and one tier alone may have ten servers—we had to dump the logs from each server separately and sift through reams of data”, the interviewee stated. In addition, a single application problem might occur as the combined result of multiple issues on multiple servers. So the information they needed could be partly in one log and partly in another.

They downloaded Splunk and appreciated its simplicity and ease of use. It solved their problem by combining information from multiple sources and selecting out the most relevant data. In their environment, outages could cost $10,000 per minute. Based on reduced downtime and reduced MTTR, this company quantified total hard ROI from the Splunk investment at $6 million and paid for the product in the first month.

To summarize, ROI is out there. However, it is hard to find companies capable of quantifying it. Last year I found four, this year I found three, and all of their stories are impressive and compelling. It’s always fun for me to talk to IT professionals who are passionate about what they do. The three people interviewed for this report definitely were, and gave us some great stories.

Enhanced by Zemanta