The sheer variety in the ways contributors organize their data could have been a major challenge to the report’s coherence – but the Verizon RISK team has dealt with this challenge in a way that not only continues to support its primary mission of delivering clear evidence of actual breach incidents, but enables the team to report on the wider variety of data as well. As the report describes (p. 9):

“In the past, the DBIR has focused exclusively on security events resulting in confirmed data disclosure rather than the broader spectrum of all security incidents. The 2013 DBIR will continue that tradition for the primary analysis, but will extend that focus to all submitted incidents in designated places throughout the report. The reason for this change is simple. We received over 47,000 incidents from our contributors in 2012, but only 621 of those were confirmed data disclosures with enough detail to produce a reasonably complete VERIS record sufficient for DBIR-level analysis.” [VERIS is the Vocabulary for Event Recording and Incident Sharing embraced by the DBIR’s authors – see the VERIS community site for details.] “We could either toss tens of thousands of records in the bit bucket, or we could “VERISize” them to the best of our ability and include them in this report. Because we like data and assume you do too, we chose the latter and highlight (separately) the larger pool of incidents throughout the text.”

Maintaining the primary focus on breaches does more than give a focus to the report. It enables VERIS to be the anchor of data normalization for this study – no small achievement, given the diversity of contributors and ways they characterize their own data.

The DBIR takes full advantage of VERIS to enable organizations to understand the factors involved in the actors, assets, attributes and sequence of actions that lead to breach incidents. Understanding the sequence of events that lead to a breach – and the points in a sequence where organizations can mitigate risk most efficiently and effectively – is one of the biggest contributors to infosec the report provides. The discussion of how the success of phishing relates to “the inevitability of the click” highlights one such example. Despite the fact that the likelihood of a user clicking on a malicious email is relative to the frequency of emails in a phishing campaign, “a user clicking does not automatically lead to a compromise. For example, a user needs to take action AND there needs to be a vulnerability on the system AND software has to be quietly installed AND there has to be a communication path back to the attacker, and, and…” (p. 38)

The DBIR highlights how and where threat actors and tactics are having an impact. By studying the report, organizations can apply findings to their own security data, and better understand how and where mitigation can best be applied.

The larger data set includes a greater proportion of large organizations this year. It is perhaps not surprising, then, that this year, the number of breaches is not so heavily skewed toward smaller organizations as it was in the 2012 report. In the 2013 DBIR, 38% of breaches affected organizations of 1,000 employees or more. It is also not surprising to find, as the report describes, a “definite relationship…between industry and attack motive, which is most likely a byproduct of the data targeted (e.g., stealing payment cards from retailers and intellectual property [IP] from manufacturers).” Other commonalities continue this year: “In a streak that remains unbroken, direct installation of malware by an attacker who has gained access to a system is again the most common vector. And that makes sense; once you own the system, it’ll need some fancy accessories.”

While organized crime remains the dominant external actor category (55% overall), the report describes how “state-affiliated actors tied to China are the biggest mover in 2012. Their efforts to steal IP comprise about one-fifth of all breaches in this dataset.” While this may be a reflection of the larger data set from a wider variety of contributors, the broader trend of evidence implicating China continues – particularly when the category is espionage, which the report describes as “state-sponsored or affiliated actors seeking classified information, trade secrets, and intellectual property in order to gain national, strategic, or competitive advantage. The only exception is when it is used for internal actors, where it refers to industrial espionage perpetrated by the employees of the victim.” At 30% of all external threat actors, China is in the lead, with Romania second at 28%. But where nearly all foreign actors are financially motivated from Romania on down, China-linked incidents are overwhelmingly about espionage (p. 22).

At the same time, however, some findings stand out – but are not necessarily as helpful. For example, the large proportion of threat actions categorized as “physical” – 35% of all breaches and 50% among large organizations – include a big chunk of attacks such as ATM skimming. While undoubtedly a contributor to data loss, this may be a less useful finding for a broader spectrum of organizations seeking, for example, to mitigate risks arising from physical access to high-value network and data center assets or physical control systems.

Other findings will undoubtedly raise an eyebrow or two. One of the advantages of the larger data set is the insight it provides into activity that may not have led to a data breach per se. A standout here is the number of threat actors among the entire body of 47,000-plus incidents identified as “internal”: a whopping 69%. This would seem to lay to rest the contention that most threat actors are external. Hark, what is that faint but growing roar? Methinks I hear the echoes of much rejoicing among the internal-majoritists – but not so fast. As the report describes, “this is an important reminder that confirmed data breaches are a rather exclusive subset amid the plethora of all types of security incidents” (only 14% of confirmed breaches were linked to internal actors). Another point to bear in mind: 48% of all threat actions across the entire 47K-plus-incident data set were attributable to error. Even among highly privileged users such as system administrators, the proportion of incidents not linked to financial or espionage motives made up the majority. (“Perhaps the slightly less breach-prone regular users should seize the opportunity afforded here to start grumbling about the “stupid admins” for a change.”)

Other findings do, however, give cause for pause. In the graphic describing breach count by discovery method and time to discovery (Fig. 45, p. 55), the big standout is the number of cases where discovery took months, and only then was made by an unrelated external party. This was the case with 5 times more breaches than the runner-up: breaches which also took months to be discovered by a customer, followed closely by breaches discovered by fraud detection – also measured in months.

It appears we are not getting much if any closer to doing a more effective job with closing the gap between gathering security data and turning it into security intelligence. But at the same time, however, consider this comparison to Mandiant’s M-Trends 2013 report called out by the Securosis guys: The Mandiant report notes that, “in 2012, 37% of the organizations we responded to discovered the intrusion themselves, versus just 6% of the organizations we helped in 2011. We also saw more than a 40% improvement in the median time an attacker was present on a victim network — down to 243 days from 416 days in 2011.”

While this may be a reflection of the greater sensitivity and commitment of those organizations to incident awareness, they are also early leaders in what I believe will become an even more pronounced trend: the value of proactive security investigation, adopted as a practice regardless whether evidence of a damaging threat is as yet apparent or not. It’s important to note that, as EMA’s 2012 report on the rise of “data-driven security” highlighted, those showing the highest maturity in investment in both the technologies and expertise of data-driven security are also among the earliest adopters of modern tools and applications of risk analytics to infosec. We also saw, however, that the majority also expect external sources such as service providers to lead the way in delivering approaches that are more accessible to a wider scope of organizations. I expect the evolution of this aspect of “Security as a Service” will continue to flourish as a very active area for providers of managed and professional services and SaaS-based technologies that can help more organizations close these gaps in visibility and response.

Among those strategies, the integration of intelligence into the tactics of defense ranks high. It is a primary factor in so-called “advanced threat defense” that combines inline detection in multiple channels (network, web, email, file systems, etc.) with intelligence gathered both locally – through heuristic techniques that may include direct observation through sandboxing – as well as across networks. The combination sharpens visibility into the nature of more complex threats such as attack networks, and can arm its users with more effective tactics for attack disruption.

In that last bit, however, is hidden an aspect of security intelligence that often goes unrecognized, or at least, not always fully appreciated for its potential impact on security architectures. It is simply this: in order for intelligence to factor into effective response, proactive defense or environment hardening, security intelligence systems must be able to send data out as well as take it in.

Simple enough, right? Okay, then, tell me: Of all of you who have deployed SIEM, log management, or operational IT monitoring for security in your environment, how many of you have these systems actively feeding actionable data to your defenses? “Actionable” in this sense isn’t just the hackneyed cliché that we tend to overuse in this business (well, it is anyway, but give me just this one). What I mean is, how many of your security data management platforms deliver this trifecta to your defense tactics?

• The nature of a specific issue, such as an attack technique or vulnerability – known and published or otherwise – detected, say, in exploitive behavior
• Consumable in such a way as to enable defenses to act directly on the information
• AND, the tactic is indeed capable of remediating the issue accordingly, through reconfiguration, blocking, isolation/segmentation, and so on (Extra credit if this is fully automated end-to-end. Okay, so that’s also overused marketing-speak. But heck, if you already have something like this working in actual production…well, you walk away with all the cheese regardless.)

Sure, there are lots of reasons beyond the limitations of monitoring technology why we wouldn’t want to do this. Automating blocking at scale would do a little more than step on the toes of IT operations and irk our insect overlords, if what we effectively build is the Mother of All Denial of Service Vehicles that raises existing problems with false positives to an entirely new level.

Still, we can’t overlook the limitations of security intelligence and monitoring tools that are predicated on taking data in. Many (if not most) are not well equipped to serve that data – or more precisely, the actionable fruits of analysis – back out into proactive defense technologies. Not terribly surprising, really, considering that they have largely arisen to inform security analysts.

And yet the market, in its omniscience, tells us we are already far along in voting with our feet for connecting this intelligence more directly to the technologies of defense.  “Why would we want, say, SIEM to do this when <your favorite technology here> already provides intelligence-driven defense as part of a comprehensive product?” The fact that <your favorite technology> is succeeding at this – and some, particularly in “advanced threat defense” are among today’s most successful security plays – should tell us something: We’re not using our security intelligence assets to the fullest. If we have a security intelligence platform of some sort, we are already taking in a wealth of information that could be put to good use in sharpening defense. Busting down the silos that keep these platforms from informing even more responsive action across a variety of defense tactics, however, may be no small undertaking.

Regardless, product vendors and service providers alike who have invested significantly in security intelligence are not blind to the success that segments such as advanced threat defense are enjoying, in no small measure due to their ability to link intelligence with action. Indeed, if anything, the need for this is only going to accelerate, as the growing capabilities of ubiquitous mobile devices expand the need for intelligent defenses that can “understand” a variety of new exposures and threat scenarios. If they don’t move in these directions, vendors who have predicated their strategies on intelligence are at risk of ceding the lead to the markets where these techniques are succeeding.

Two years ago, I speculated that the rise of data-driven security tactics would be one of the transformative factors shaping security in the future. We have already seen major companies recognize this in acquiring data-driven defenses, from inline fraud and threat mitigation to systems management platforms with a high degree of flexibility in the nature of data they can consume (even if they may not yet be utilizing this flexibility to the fullest).

I expect to see this go even broader, particularly in increasing the output from security intelligence systems into a range of defenses that can consume this intelligence directly, from proactive environment hardening to more automated approaches to threat mitigation and incident response. Expect to see further vendor announcements along these lines this year.

Just as prevalent at RSA are the many claims from a host of vendors about the latest can’t-do-without tool or widget that will solve all our unsolvable security problems (problems that often serve us equally well as FUD fuel, by the way). Security is always a fairly noisy market, with never any shortage of opportunity to chase a buck when attacks remain so prevalent – and successful.

But what all this noise reveals is a serious and longstanding fragmentation of the IT security industry. Competition can be a good thing, when disruptors challenge incumbents and keep everyone’s game sharp. But what the security market often shouts louder than anything is just how little real integration exists among security tools.

Not that this is all that unique to security, certainly. But the nature of security concerns today demands better coordination of insight with response. Prevention may be pointless if tactics are blind to current realities or simply outmoded. Threat and vulnerability awareness must be complemented by strengthening of the security posture in ways that will make a real difference. Those responsible for security strategy must know where they can best place their bets.

The adversary often knows these gaps better than we do. The so-called “hybrid” attacks of a decade ago that combined vulnerability discovery with exploit and self-propagation payloads heralded what have today become highly efficient attack toolsets. Complex attacks often combine social engineering with malware communicated via multiple channels, hosted by endpoints and servers alike. The more dedicated adversary, meanwhile, predicates success on stealth and the ability to obscure malicious activity under the guise of normal behavior.

This would seem to call for a more coordinated approach to defense – and yet, vendors in silos trample merrily all over each other, proclaiming that their approach is better than that of others, with seemingly little regard for the consequences.  The result? “Poor integration among security tools” was one of the top two frustrations with security technologies reported in our 2012 survey of 200 enterprises worldwide.*

There are, however, some key areas in which integration is beginning to play a role in product evolution:

• Mobility can be expected to be a hot topic at RSA this year. Already within the mobile management market, mobile device management (the management of device policy) and mobile application management (app isolation and control) have come together in some vendor offerings, while mobile management itself seems poised to align with existing approaches to enterprise endpoint management. Network Access Control (NAC), for example, has already pioneered concepts of pre-admission device assessment and isolation of network traffic. The alignment of NAC with mobile management is already evident in partnerships such as ForeScout’s with AirWatch, Fiberlink, MobileIron and others. But there is one aspect of mobile security that I have yet to see much discussed, yet which could hardly be more important to mobile security strategy…and it’s an aspect of the following:

• Security intelligence was one of the more overheated topics at RSA last year, with the term “Big Data” being flashed like a high-stakes gambler’s cash. But as I have described at some length, there are some very good reasons for this trend. Alleviating the blindness that has long beset security strategists is just one aspect of data-driven security’s promise. Defense technologies that integrate insight and analysis directly into their functionality are another provocative example, such as RSA’s recent acquisition of Silver Tail Systems, and Juniper’s of Mykonos. As data-driven tactics become more evident, I expect the integration of intelligence into defense to continue to play a larger role in shaping more effective security technologies – including some examples not yet prevalent, but which, I believe, will be.

• Consider, for example, the risk posed by a plethora of apps for today’s mobile devices. Device control and app segregation can do much to mitigate mobile risks and insulate enterprise resources, but the threats posed by promiscuous or poorly behaving apps themselves is already a big concern given the thousands of apps available in the consumer market. Researchers have already demonstrated how potentially malicious apps can get through trusted app stores. Developing mobile app reputation intelligence on a scale at par with the sheer numbers of mobile apps would be no small matter for anyone willing to take it on – but in the future, it could be a valuable complement to be integrated with the mobile security offerings you’re likely to see at RSA next week. You can expect your preferred mobile security and management vendors to be taking a close look at companies such as Appthority who have made some inroads in this direction.

• Intelligence is already a primary factor for one of today’s poster children for integrated security tactics: so-called “advanced threat defense,” which will be much in evidence on the RSA show floor. FireEye may have been a pioneer here, but the approach has become embraced by nearly any vendor that today includes the word “fire” in a brand, as well as by some who don’t. This approach correlates activity monitoring with intelligence such as malware recognition or interaction with known malicious sources, complemented with the ability to monitor, isolate or block activity in some cases. Because complex threats often leverage a combination of techniques, (phishing to gain an initial foothold, for example, followed by the surreptitious download of malware or other attack tools, often via compromised websites), these “advanced” defenses often feature coverage for multiple avenues of attack, across email, web or enterprise content repositories (where threats may often be stored unawares as a result of such attacks).

These examples are promising, but they still do not address the many challenges of weaving a coherent strategy out of a plethora of many parts. There are some initiatives seeking to tackle this challenge head-on:

• SCAP, the set of specifications that make up the Security Content Automation Protocol, has been advanced in government as a means to get security tools to play well with others. The effort is ambitious, since it embraces a number of specifications for defining how security tools interact with each other, which must be kept current with both technologies and threats. Vendors often resisted, not least because developing to SCAP specifications often meant duplicating development costs when existing formats were already in place. But there are some in the private sector such as Ed Bellis, now CEO of Risk I/O, who have seen the promise of SCAP pay off in real benefits, while federal mandates have helped to drive adoption. Largely because of the latter, you can expect to see at least a few announcements at RSA that speak to the adoption of SCAP in government.

• “Software-defined” technology - Software-Defined Networks (SDN) and the Software-Defined Data Center – may herald the future of such trends, offering greater flexibility for security as well as for network management. Some vendors such as Juniper, have placed significant bets on this promise, aligning with a focus on complex or large-scale enterprise and service provider environments which often make extensive use of virtualization. The potential of such trends for offering security a systematic way to define policy throughout an infrastructure is tantalizing, but it is still early in terms of on-the-street products. Regardless, this is a concept I expect to see making more of an appearance at RSA this year.

Despite such initiatives, examples of real integration today are far too isolated and rarer than they should be. Will vendors at RSA be talking up the values of an integrated approach, or how they interoperate with other techniques? Heck, I’d be thrilled if they would at least talk more publicly about cooperating with each other in sharing intelligence about the threats to the industry as a whole. Am I overly optimistic that security toolset integration will ever become more evident? Probably. But that doesn’t mean the need doesn’t exist.

Gunnar Peterson has offered some excellent advice for RSA attendees: keep an eye open not so much for the latest bells, whistles and hype, but rather for how products and services will actually fit in your organization. To this I would add, be on the lookout for vendor trends toward real product integration – and don’t be shy about pressuring your suppliers to do a better job, both in integrating their own products as well as in working with others, to solve the larger problems we all face.

*At 43% of all respondents, poor integration among security tools was nearly tied with the most frequently reported frustration, “too difficult to distinguish which security actions are working, and which are not” (46%).

In the wake of last week’s disclosure of an attack against Bit9, Jeremiah Grossman seems positively prescient. His New Year’s prediction about security’s immediate future was that attacks against security measures would increase. And why not? If defense is vulnerable, why attack other weaknesses? A takedown of defense could lay a much larger and richer target open to compromise.

Therein lies the key: leverage. An attack that yields far greater results than defeating the immediate objective is using leverage. Minimum effort here yields comparatively much larger results there.

I still don’t think we fully understand what that means.

I applaud Jeremiah for calling out security measures as a likely target – but why should that necessarily mean that an adversary would target the defenses of just one organization? If leverage is so effective at exposing a single entity, why not target an attack against an asset whose compromise could expose so many others?

This is why it’s not just security products that are becoming high-priority targets. It’s the capabilities that, if successfully compromised, expose anyone who relies on them.

We have seen the precursors already, in attacks against RSA, certificate authorities, and others whose assets protect hundreds or thousands of organizations and millions of individuals. Bit9 is only the latest. There will be others. The objective is not to compromise the victim organization. It is to compromise the defenses of many, many more.

This should be a wake-up call to the security industry – but here, too, we may not really get it…yet.

It certainly means that any provider of leverageable capabilities must step up their game – because compromise for them is a matter of when, not if:

• The security market includes those who provide tools and guide practices for improving the security of applications. How many apply those tools and practices to their own products and services?
• Intelligence has become a watchword in the security industry in recent years. How many providers focus their efforts on gathering and responding to intelligence regarding their own operations?
• Response itself has become one of the most glaring defects of the security industry. Bit9′s response demonstrates how the industry is progressing. More providers should learn from these lessons in preparing for a crisis and responding in a planned and demonstrably beneficial way.
• Security vendors are not the only ones likely to be targeted in a leverage attack, however – as our ongoing Java woes should make clear. Enterprise IT management system vendors, for example, should take note: How could a compromise of highly sensitive control or defense functionality in your products – or (more likely) a lack thereof – be exploited to attack hundreds or thousands of your customers? What should the impact on security vendors be telling you about how you should prepare? Because you are a target, too. Not will be. Are.

But how do these targeted organizations determine how much investment in better response is enough? This is not just the tired old security-spending-will-sap-our-ability-to-compete dodge. If the impact of a successful leverage attack can be measured in the downstream impact on hundreds or thousands if not millions of others rendered exposed, how can any one organization marshal more assets than it has, no matter how much it could spend?

If the industry is under attack, it’s the industry that must respond. Security vendors who seize upon the opportunity to pile on a compromised competitor have not learned the lesson of leverage yet. It seems just a little ironic to me that we are often the ones calling on others to be more forthright in championing breach disclosure, sharing intelligence and improving response among peers, when we enthusiastically shoot our own wounded.

Perhaps it’s time we took a little of our own medicine – because if we don’t, regulators will most assuredly try to find some way to force it down our throats.

Related articles

• Security Firm Bit9 Hacked, Used to Spread Malware (krebsonsecurity.com)
• Sharing Intelligence (Bit9 CTO Harry Sverdlove – blog.bit9.com)
• Karma is a Bit9h (securosis.com)
• Ich Bin Ein Bit9er (ashimmy.com)
• All up your bitness (Wendy Nather’s idoneous-security.blogspot.com)

In my last post, I noted that I expect intelligence and the coordination and integration of defense technologies to be key drivers of the IT security market in 2013.  This is not just about some major vendors who have acknowledged their need for a wake-up call on these fronts. These are larger trends that are already influencing the direction of the market in multiple ways.

Over the last couple of years, I have written at length about one of these trends in particular: the rise of what I’ve been calling “data-driven” security. As the RSA Conference approaches, announcements will begin to emerge thick and fast from security vendors between now and late February. And as the trend toward a more intelligence-driven approach to security continues to accelerate, I expect those with a significant stake in data-driven security to be revealing their new initiatives in the next several weeks.

Among the most visible will be those who have focused their strategies and market messages on “security intelligence” or “security analytics” as product offerings. This includes some of the vendors most active in the Security Information and Event Management (SIEM) market – though this trend certainly isn’t limited to SIEM (as I’ll elaborate below). In particular, I expect some of these vendors to emphasize the role that Big Data will be playing in their directions this year.

This isn’t new, of course. Big Data was one of the most visible buzzwords at last year’s RSA Conference. What will be new this year will be the extent to which some of these vendors will be integrating – or announcing support for – data management platforms such as Hadoop and expanding their analytic capabilities.

Why Big Data in security products? As my colleague John Myers describes, Big Data may not always be big…but it is often complex. Designed for a distributed approach that leverages parallelism for better analytic performance of large or diverse bodies of data, platforms such as Hadoop and NoSQL environments exemplify a highly valued capability of modern data management techniques: the ability to ingest a wide variety of data with less need to force a particular schema on data at ingestion, or other strictures that make it difficult to make the most of the variety of data that security teams would pursue if they could. This ability to embrace data variety is one of the “three V’s” of big data that has particular appeal to security investigators and strategists alike: greater flexibility in discovering evidence that adversaries would prefer remain hidden in security-relevant data, as well as how and where security strategies can be optimized. The ability to unite analysis across silos of security data may be another promise of Big Data platforms that could yield new learnings often stifled by legacy approaches.

Product vendors who focus primarily on tools for enterprise security analysts and strategy managers are likely to be among the most visible making announcements this year, since these are the ones most likely to “productize” the integration of Big Data platforms with security data management. Many of these have their roots in SIEM and related markets such as log management, IT operations data management, and similar technologies. Others have focused on the administration of security and compliance efforts, such as so-called “GRC” (Governance, Risk Management and Compliance) tools. More recent entrants include those that provide security and forensic analysts, technical investigators and incident responders with deep and detailed evidence.

In many ways, the embrace of “Big Data” platforms is simply the next step in the evolution of these technologies, suggesting the likely hybrid approaches to integrating Big Data platforms with existing security data management that reflect the larger trend of hybrid data ecosystems described by my colleague Shawn Rogers.  The potential of this trend could well take these vendors beyond their current boundaries, however, introducing new approaches to data-driven security that may in part arise from the breakdown of the silos that often isolate these product categories from each other.

Some of these vendors stand out, because of their present weight or disruptive influence in the enterprise market, and the bets they have made up to now in security as well as in data management and analytics. Most of these have either made allusions to the role of Big Data in their efforts before now, or have already announced support for, or integration with, platforms such as Hadoop and/or partnerships with highly visible companies in that world. Some examples:

Not surprisingly given EMC’s primary focus on enterprise storage and content management, the company’s RSA Security Division has been able to build a competitive security data management portfolio. From the deployability of the enVision SIEM product acquired with Network Intelligence in 2006, RSA has since added strength in GRC with Archer and in investigative platforms with NetWitness. These technologies complement the company’s assets in anti-fraud, intelligence and data security, as well as RSA’s long history in authentication, and its even longer history in encryption. Beyond the RSA Division, EMC’s investment in Greenplum exemplifies the company’s commitment to data warehousing and analytics. It is the NetWitness platform, however, that most visibly represents EMC/RSA’s commitment to investigative security analytics. Meanwhile, recent acquisitions such as Silver Tail Systems represent the company’s forward-thinking view of the value of analytics engaged in real-time defense. RSA executives have been very clear about the importance they attach to Big Data to the evolution of security. I expect this importance to become even more manifest this year.

HP retains a strong hold on the SIEM market as the owner of ArcSight (not to mention competing head-to-head against the likes of IBM in application and network security). But will HP see a priority need to integrate Hadoop or NoSQL platforms with security data management products? In the last several months, the company has emphasized performance in introducing ArcSight technologies such as the Correlation Optimized Retention and Retrieval Engine (“CORR-Engine”) which (as of mid-2012) touted 10-to-1 data compression and the ability to manage up to 45 terabytes of data per instance. Vertica, meanwhile, exemplifies HP’s current investment in data warehousing and Hadoop integration, while the company boasts a range of partners and services focused on Hadoop and Big Data. And in just the last few days, HP announced new services for security breach management and forensic readiness.

IBM is the largest overall product company to take a dominant role in security data management – and this is saying something for a company who, slightly more than a year ago, went from having little more than vestigial capabilities in product segments such as SIEM to owning one of the most dynamic rising stars in the field in Q1Labs. But IBM should keep – and hold – your interest for more than security. With the acquisitions of Netezza and Cognos and the emergence of its InfoSphere BigInsights platform, IBM’s larger strategy is squarely centered on data management and analytics at scale for the “smarter” enterprise. In addition, IBM has a strongly competitive Managed Security Services organization which, when combined with the high priority it places on research generally, arms the company with considerable depth in where and how to apply its capabilities. Examples of where research in data management and analytics are taking IBM go beyond the company’s most visible PR spokesperson – or more correctly, “spokesthing” – namely its Watson project, and include initiatives such as its InfoSphere Streams technology, which has provocative promise for the analysis of data in motion.

Splunk has become a disruptor in this field, taking the technologies of search into the realm of what it calls “operational intelligence” and opening new vistas in terms of flexibility in finding and using a wide variety of operational data in an intriguing diversity of use cases. It must be remembered that search, which is fundamental to Splunk’s success, and today’s emerging technologies for managing diverse data at scale, both arose from the same origins. Recently, this evolution has come full circle at Splunk, with the introduction of Splunk Hadoop Connect. Splunk has also introduced the Splunk App for Enterprise Security as part of its security strategy, challenging incumbents in the SIEM market and providing an alternative for organizations frustrated with products that seem forever to tantalize more than fulfill the promise of SIEM. Splunk’s capacity for disruption was amply evident in the company’s IPO this past spring, which raised over $3 billion on its first day. Though the stock has fluctuated since, it hit highs around the time of the company’s annual user conference in September when positive customer sentiment was in evidence (based on our own experience at Splunk .conf2012).

Of these four, EMC’s RSA Division excels in two areas where its competitors face significant challenges: the NetWitness investigative platform, and Archer for security and compliance management. RSA also has a well-developed portfolio of anti-fraud solutions that appeal to banks and financial services – now augmented with Silver Tail defenses for applying analytic techniques to fraud or application abuse detection in real time.

IBM, on the other hand, brings strength not only in SIEM with Q1Labs but in other aspects of security such as Web applications and network defense, while both IBM and RSA have somewhat different stakes in identity. More significant in the long run may be IBM’s business analytics and data management capabilities beyond security. This not only enhances the company’s credibility in the C-suite, it may also be significant to achieving an important goal of data-driven security: the objective identification and management of security priorities, and communicating the value and effectiveness of security measures to the business. Today, RSA can answer with strength in fraud intelligence and defense and a well-accepted platform for managing the administration of security and compliance efforts with Archer. In the future, however, EMC may have to go farther to demonstrate just how important head-to-head competition in analytics beyond security – or beyond Greenplum – will become. Meanwhile, both HP and IBM have much ground of their own to cover in order to become more directly competitive with both Archer and NetWitness, suggesting the opportunity for vendors such as AccessData and Solera Networks and the expertise of Mandiant.

With its ownership of ArcSight, long dominant in SIEM, HP has also faced significant challenges in the frustrations that many have expressed with security data product deployment, ongoing maintenance, and the limitations of legacy. Breaking down these limitations has been key to the success of SIEM disruptors such as NitroSecurity, whose acquisition by McAfee was announced at the same time as IBM’s of Q1Labs, and emerging leaders such as our neighbor here in Boulder, LogRhythm, which has introduced a highly usable and intuitive approach to security-relevant operational data in what it calls “multi-dimensional” analytics.

But technologies such as Hadoop hold promise beyond SIEM – and even within the realm of SIEM and log data as it is today, vendors such as Splunk and LogRhythm should be watched for what they can do that others cannot – or at least cannot do as well, or in the same way. The flexibility of the Splunk approach has not only led some to actually abandon their efforts to try and make the most of SIEM (as at least one Splunk customer eagerly described to us at .conf2012). It has also introduced a degree of freedom in data exploration that has – as recently as this past week – led more than one vendor to mention Splunk or “Splunk-like” capability in their analyst pitches. True, Splunk mastery may mean an investment in its distinctive techniques such as the Splunk search language – but this investment often yields significant returns when it unleashes flexibility that can be directed toward a very open-ended range of possibilities. Where Splunk could stand to benefit is in an increased investment in technologies such as data visualization.

Which suggests where future directions among all these contenders and others may lead. Each of these vendors has emphasized some degree of commitment to “Big Data” – but fundamentally, that means the management of data scale, speed or diversity. Where data-driven techniques may make an even greater impact is in the expansion of analytics that make evidence come alive. As the “backend” becomes more adept at velocity, volume and data variety, security analytics and data visualization will need to mature not only to communicate intelligence rapidly and effectively, but to free organizations from the dual challenges of finding and retaining both qualified security expertise and qualified data scientists – and neither role is particularly easy to fill today. Innovators such as Packetloop or PixlCloud may have a hand in showing the way forward, reflecting the success of those such as Datameer in broader analytic markets, while experts such as Mandiant continue to blaze trails in security investigation and data management through support for initiatives such as OpenIOC.

Longstanding security vendors such as McAfee, meanwhile, challenge incumbents with assets such as NitroSecurity. But vendors such as McAfee, Symantec and Trend Micro, as well as managed services powerhouses from Dell’s SecureWorks to Vigilant and vendors of “advanced threat defense” such as FireEye and Sourcefire, may be even more interesting for the directions in which they take their services that are “backended” by Big Data. Symantec, for example, did not articulate many specifics regarding future product directions in last week’s announcements of its new strategic focus – but one provocative hint that it did drop had to do with where it might take is current investments in security intelligence, possibly making them more directly available to enterprise customers. Consider as well the potential of vendors such as Akamai, whose visibility into application traffic worldwide give it a distinctive view into security. These suggest the strengths that all these vendors – who already collect enormous amounts of threat intelligence – can capitalize upon.

All of which should make for an interesting next few weeks in the runup to the RSA Conference, as the power – and value – of data and analytics continue to change the nature of security insight.

Related articles:

• Big goals for Big Data (networkworld.com)
• Security Analytics: Hype or Huge? (infosecprofessional.com)
• Five big data predictions for 2013 (strata.oreilly.com)
• HAWK Network Defense Joins Sourcefire’s Technology Partner Program (prweb.com)
• RSA is betting its future on Big Data (networkworld.com)
• Flying Under the Radar: Insurance Claims Fraud Detection (bigdata.pervasive.com)

Two overall trends exemplify some of the more visible aspects of threat evolution. As common exposures go unresolved, the packaging of commoditized attacks becomes what may be fairly qualified as “industrialized.” At the other end of the spectrum, the more dedicated adversary learns how to adapt a sequence of activities – many covert, but some neither particularly stealthy nor “advanced” – to targeting high-value objectives.

Note these recurring themes that have played a role at either end of this spectrum:

• Intelligence matters. Adversaries collect it to better understand the target and identify opportunity. Defenders need it to shape effective response.
• Coordination is also important. A logical sequence of activity is a hallmark of the more dedicated adversary, while the packaging of multiple and often complementary attack techniques has become a signature of industrialization. On the defense side, however, the coordination of defenses and response has too often been lacking, which presents an all too reliable opportunity to the adversary.
• Intelligence and coordination are often flip sides of the same coin. Understanding the nature and context of attacks and how threat activity plays out are key to getting a better handle on intelligence. The coordination of response, meanwhile, depends on accurate insight into where and how response can best be applied.

Closing these two gaps – in intelligence and in the integration of defense – are becoming primary factors in shaping the security market. We have certainly seen these objectives emphasized in 2012. I expect these emphases to intensify in 2013.

“Hold on there a sec,” you say. “I thought you said intelligence and coordination.” “Integration” isn’t the same thing.

True enough. In one sense, coordination means getting better at being nimble – and security has not exactly done well at being nimble. The fact that, for example, 92% of breaches in the caseload reported in Verizon’s 2012 Data Breach Investigations Report were discovered by a third party (up 6% from the prior year), is testimony to that.

Regardless, one definition of coordination means (according to Wiktionary), “making different people or things work together for a goal or effect” – and this is has been a classic failure of security technologies. Prevention means a stronger defense capable of withstanding compromise – but far too often, we have failed to even so much as recognize the nature of compromise, to the point where prevention is no longer considered nearly as important as awareness and response. In fact, many of us are already so compromised in the strength of the posture we present to the adversary that we enable the industrialization of commoditized attacks. Meanwhile, our blindness is exacerbated not only by not taking advantage of a wider breadth of relevant information, but also by not making good enough use of the information we already collect. Adversaries of all kinds have exploited these weaknesses successfully.

Part of the problem is the nature of the security technology market, which has long been characterized by new widgets that arise in almost a knee-jerk reaction to the Latest Hot Thing. If the Thing is Hot enough, a bunch of widgets compete with each other for the same buzz (and beg us analysts to declare them a Market Segment). And as with many venture plays, a key objective of many of these startups is exit through acquisition. But acquisition of the Latest Hot Thing without a vision for how it integrates into a greater whole has caused problems for some acquirers, who seem at times only to have seen the opportunity to grab markets and revenue at the expense of long-term effectiveness.

When it comes to informed and effective response, this sort of approach is doomed to failure. Keeping intelligence and response tactics isolated and dysfunctional is one of the biggest overall exposures the adversary can exploit. (It’s also an aspect of the immaturity of many approaches to security product development, which is itself a problem that may lead to bigger issues in the future, as Jeremiah Grossman suggests.)

This, then, is why I believe defense must embrace a more mature and integrated approach – and it’s not that I’m particularly Pollyanna (or even optimistic) about this, given how poorly we’ve done with such efforts in the past. But I also believe this past failure is a good deal of the reason why integrated defenses are already making their appearance. It is an aspect of technologies that target “advanced” threats that are making a significant impact on the security product market – and which will continue to do so as intelligence becomes an ever-greater aspect of integration – while “next generation” defenses should, by definition, attack these weaknesses (as certain aspects of securing more capable mobile devices already do).

These are big goals that we’ve had as an industry for a long time – but the fact that “too difficult to distinguish which security actions or policies are working, and which are not” and “poor integration among security tools” were the top two frustrations with security technologies reported by working professionals in our 2012 research, suggests (to put it, um, gently) that we have a long way to go.

With these factors in mind, I’ll take a look in upcoming posts at how the integration and coordination of techniques are shaping new and emerging approaches to defense, and where “data-driven security” is headed today – beginning with my next post, in which I’ll take a look at the major players bringing “Big Data” and modern analytics to bear on security intelligence.

I’ve written about Silver Tail before, going back to one of my first posts on the rise of data-driven security. In a five-part blog series introduced by that post, I described how data-driven security is evolving in three primary domains:

• Data-driven defense tactics which differ from legacy security technologies in their focus on a more continuous, dynamic dependence on data sources.
• Data-driven security strategy and management that emphasizes objective techniques for yielding more effective insight from large amounts of many different kinds of data, and
• Data sources to serve these interests, which can be expected to grow in their variety, as well as in the variety of ways they are made available as offerings in their own right.

Silver Tail offers an exemplary demonstration of emerging data-driven defense. Its technologies analyze the behavior of interactions with websites and web applications to understand expected behavior and identify deviations that indicate fraud or malicious attempts to abuse business logic. This requires the fundamental engagement of data analytics to identify norms and recognize anomalies – an application of analytic techniques that are already widely familiar in realms such as Business Intelligence. They also play a role in the analysis of security and fraud data – capabilities that are growing in visibility, in concert with growing awareness of just how broadly data analytics can be applied.

Up to now, the concept of data-driven security has been largely associated with security management, exemplified by technologies at the heart of the Security Operations Center, such as Security Information and Event Management (SIEM), investigative and forensic analysis platforms, or IT GRC platforms for rationalizing compliance requirements or managing vulnerability remediation (all capabilities already present in the RSA portfolio, by the way).

Silver Tail represents a category of technology that takes data analytics out of the SOC and puts them directly to work “in the field,” on the front lines of tactical defense. Like other data-driven defenses, Silver Tail analyzes observed activity in IT systems. Silver Tail targets the ways malicious parties interact with business applications “in a different way, at a different speed,” as company CEO Tim Eades describes it.  Like domains of web analytics beyond security, it employs large-scale data management to derive this insight – as many as 300,000 clicks per second according to Eades.

Where Silver Tail stands out is in its ability to bring these capabilities together in real time defense. The analysis of real-time data enables Silver Tail to build models of behavior and leverage machine learning to differentiate customers from criminals. This means it has much in common with RSA’s Identity Protection and Verification (IPV) Suite, complementing a number of anti-fraud intelligence and adaptive authentication capabilities in the IPV suite with real time detection and exploit prevention.

Silver Tail offers both software and SaaS deployment models that do not require the instrumentation of Web applications or deployment of client-side software. The SaaS model in particular suggests the high promise Silver Tail holds for adding real time fraud and threat analytics to cloud-hosted security tactics, complementing approaches to Web security “as a service” exemplified by Blue Coat’s Cloud Service or Zscaler. RSA foreshadowed the value it attaches to this synergy this past February, in a partnership with Zscaler to augment its services with RSA Adaptive Authentication and Cloud Trust Authority. A hosted approach aligns well with the large-scale data management platforms that can extend the value of Big Data techniques to security for a wide audience.

In EMA’s 2012 survey of the rise of data-driven security among 200 enterprises worldwide, 40% of respondents reported that they are overwhelmed with the security data they already collect. 46% have insufficient time or expertise to analyze what they collect. At the same time, however, 73% said that they would collect even more security-relevant data, if they could make use of it.

Silver Tail represents a growing class of technologies meeting a strongly demonstrated need, making insight available as real-time defense derived from large bodies of behavioral data. Its acquisition by RSA signals the growing validation of data-driven security tactics, as enterprises – and the vendors that serve them – recognize that effective analytics are the key to understanding the evidence of threats that too often escape detection.

“But you’re a security analyst.  Why are you going to IOD?” you ask. (And if you didn’t, let me explain:)

With over 12,000 attendees, IOD is a major user event in the realm of information management and data analytics. If you’ve followed my research for the last several months at all, you begin to understand the connection.

In a report I published this past spring, we saw how organizations are showing high interest in expanding their ability to turn security-relevant data into insight and action. They recognize that, without this insight, they are fighting blind against an enemy that has the advantage in choosing any number of targets in complex IT environments. Among the most significant contributing factors:

• An increased recognition among organizations that high-value information is the target of an adversary that may have already penetrated defenses, whether the business is aware of it or not.
• Recognition as well that even trusted organizational personnel and partners can increase security risks, either intentionally or unwittingly through errors or exploits of user behavior.
• A growing desire to do more than make the most of intelligence sources, both internal and external. Organizations seek to make intelligence more actionable, correlating insight when possible to actual exposures and activity.
• The need to make the case for security strategy and investment to executives and organizational management, in line with the adoption of analytics among other aspects of the data-driven business.

How do these objectives relate to IBM? Consider how the company has parlayed its legacy success in the mainframe into substantial investments in data management, data warehousing and analytics. It is today a force to be reckoned with in these fields, posing one of the most significant competitive threats to dominant players in these markets such as Oracle.

At the same time, IBM has built a portfolio of security products and services that, in some markets, rival leaders there as well. But it has not always been so. While IBM has long held a strong position in technologies such as identity and access management, it was not until its acquisition 6 years ago (this past week, in fact) of Internet Security Systems that the company made a more direct assault on IT security. IBM later added leadership in application security to its portfolio with Watchfire and Ounce Labs, while its acquisition of BigFix brought security benefits in systems and endpoint management. With its 2007 acquisition of Princeton Softech, IBM brought Princeton’s Optim data masking and privacy technologies to its portfolio. IBM’s stake in security for information was further augmented in 2009, with Guardium and database activity monitoring.

It was not, however, until its acquisition of Q1Labs last year (also this week) that IBM closed a substantial gap in one of security’s most significant markets: Security Information and Event Management (SIEM). With Q1Labs, IBM gained one of the most successful challengers to the field’s incumbents, including HP’s ArcSight acquisition. This has given IBM yet another leadership stake in security, and one that is arguably foremost in the minds of many practitioners for its role in unifying the collection and analysis of data from a host of point products – defenses as well as IT systems and management technologies.

But SIEM is not the only aspect of information management and analytics that enterprises find important to security:

• In recent years, many have grown to appreciate the value of investigative platforms such as AccessData’s SilentRunner, EMC’s NetWitness, Guidance Software’s EnCase portfolio, Solera Networks and others that capture forensic evidence and provide the deep and detailed insight needed to recognize the nature and activity of specific attacks.

• SIEM is not the only tool used to wring meaningful security insight from disparate data sources. For example, a class of tools such as Core Security’s CORE Insight, Firemon’s acquisition of Saperix, RedSeal Networks, Skybox Security’s Risk Control – and IBM’s own Q1Labs QRadar Risk Manager – collect security data from a number of sources and seek to identify the most significant exposures and possible opportunities for attack. Because of this emphasis, these technologies have often emphasized risk management as a central aspect of their value.

• Risk management, however, may mean little unless risk can be expressed in terms – realistic terms – of business impact. IT GRC (Governance, Risk Management and Compliance) platforms have made a stake on this objective, but for the most part they have focused largely on unifying enterprise efforts to comply with an often dizzying list of regulatory demands and providing a means for defining processes to establish objectives. When IT GRC solutions deal with IT risk, it may be limited to vulnerability management in its conventional sense: correlating software or configuration defects with IT resources and tracking remediation. In other words, they have often emphasized the G and the C in IT, but too often stop far short of risk management as other aspects of business understand it. Still, the “top down” nature of GRC (defining objectives and priorities at a high level and tracking performance in meeting those objectives in operations) could hold promise for achieving a goal difficult to realize from a bottom-up approach to infrastructure vulnerabilities: the ability to understand the value of complex IT assets to the business, and prioritize risk management measures accordingly.

Which brings me back to the Information on Demand conference. IBM has strength among its assets – information management and analytic assets, many of which are not yet seen as security “players” – that could be brought to bear on much, if not most, of everything I’ve described:

• As enterprises continue to see the value of gathering – and retaining – security-relevant data from a number of diverse sources, they also see the value of both current and emerging technologies for managing large and diverse data repositories. IBM is already a serious competitor in data warehousing with its Netezza acquisition, and its InfoSphere business is building out a credible platform for highly distributed data management through its investments in Hadoop and NoSQL techniques – but that’s not all. I have repeatedly heard security organizations discuss the potential for technologies such as streams for analyzing data in motion, bringing insight closer to real time than technologies that must centrally collect sets of data in order to analyze them. IBM’s InfoSphere Streams initiative could have substantial promise in security if it matures to meet these expectations.

• Some have criticized so-called “Big Data” initiatives in security as collecting far too much data, in pursuit of – well, of what, is not always clear. Or so say naysayers. Tell that to the banks, government agencies, healthcare enterprises, and others who are making substantial investments in Big Data technologies and expertise…for security. Many, if not most, of these organizations may tell you that they don’t see these investments as a replacement for technologies such as SIEM. Indeed, most that I have spoken with are clear that, while SIEM continues to have high value in monitoring security and triggering events, the need is equally great for maintaining as complete an inventory of historical data as possible that can be analyzed to understand exactly how and where the enterprise faces exposure and sustains threats. The world of data management and analytics offers an analogy: online transaction processing (OLTP) for the “real time” view on which day-to-day-operations depend, versus OLAP – online analytic processing – for understanding historical data that can inform current trends and support decision making for the best future outcomes.

• In its OpenPages and Algorithmics acquisitions, IBM has brought together a provocative pair of assets that could give it a well-differentiated edge in risk management for IT. The OpenPages platform already serves many organizations with enterprise GRC capabilities. Algorithmics, meanwhile, provides risk analytics that are familiar among actuaries and other business risk professionals. These professionals typically have little if any involvement in IT management – but this could be a good thing. One of the areas in which IT risk management struggles is in identifying criteria that can be measured effectively. Algorithmics represents the very sort of probabilistic analytics that could make a difference in understanding not only IT risk, but its relevance to operational risk management.

• This is not all that IBM could offer in the way of risk analytics, however. As mentioned earlier, a “top down” view of IT assets could build a better view of IT asset value relevant to understanding the business impact of IT risks than a sum of bottom-up details. Identifying how IT resources fit together in unified business services would much better identify where exposures at the detailed level of infrastructure could have the greatest impact. IBM has additional assets beyond Algorithmics and OpenPages that would answer this need in IT, such as TADDM – Tivoli Application Dependency Discovery Manager – which IBM gained with its acquisition of Collation in 2005. TADDM discovers the components of an application and enables IT operations teams to understand how complex applications are built and operate, revealing critical dependencies that may otherwise go undiscovered by other techniques. TADDM’s application focus could complement assets from OpenPages to the AppScan family and QRadar Risk Manager to form the nucleus of a new and distinctive approach to IT GRC that competitors would be challenged to match.

• There is one other aspect of “Big Data” that is often overlooked in oohing and aahing over its promise, and that is the security of large, diverse or fast-moving bodies of data. But here, too, IBM has recognized its potential, particularly with acquisitions such as Guardium and Optim, which are now being applied to monitoring and securing emerging data management technologies.

These are all things IBM could do. The big question is, will it? There’s little doubt that IBM has the heft and gravitas in the corner office to be a trusted partner at the highest level of some of the world’s largest organizations. Added to this is a highly competitive managed and professional security services organization that can meet some of the most challenging objectives of major enterprises.

But whether or not IBM’s larger information management and analytics business recognizes these opportunities in security – or conversely, whether IBM’s security products and services organizations see the potential in this aspect of the greater IBM – remains, in some respects, to be seen. Many organizations still have their hands full simply fighting security fires. Today, the strategic management of security-relevant data marks out enterprise thought leaders. Will data-driven security become more common, as the technologies and expertise required come more within the reach of a wider spectrum of organizations?

It is evidence of this potential that I will be looking for above all at Information on Demand this week. Does IBM’s information management and analytics business recognize its many opportunities in security – either in its own right or as a vital aspect of operational risk management? To what extent does IBM’s growing security products and services businesses (or their customers) see the data-driven opportunity? And more importantly, to what extent are they actively working with IBM’s information management businesses to bring this opportunity to fruition?

Hopefully, IOD will offer at least a glimpse into a future where security and IT risk management are not exceptions, but are instead active participants in an evolution that IBM CEO Ginny Rometty described only a few days ago:

The next era, for all jobs, not just those in computing, will be to help businesses sift big data, she said. “It’s going to be a whole era to help you get through this, and everyone’s job, from chief marketing officer to chief of police…[is] going to be redefined by that”…

Now that the report is published, what we’ve discovered over the course of the past year-plus should be of interest if you are one of the many who feel that lack of awareness stymies security efforts, regardless whether in security operations, in the technologies of defense, or in managing security strategy.

If you have followed my blog series on this trend (and posts since), you have the general outline of the study. In the research report, however, I go into greater depth on the topics covered in the blog series, including:

• The drivers motivating organizations to look to a wide variety of data sources to improve security tactics and get a better handle on strategy. (Hint: The blind should no longer lead the blind.)
• Security’s challenges of ETL, the extraction, transformation and loading of data (a maddening frustration we in security tend to minimize with pretty euphemisms like “data normalization”), and how data management technologies can help overcome one of the core obstacles to making more effective use of security data.
• The varieties of techniques organizations are exploring to derive new understandings from security-relevant data.
• How technology vendors and service providers are expanding capabilities through data-driven approaches.
• The role of technologies such as data warehousing, optimized data management techniques such as columnar systems, emerging NoSQL environments, data mining and analytic tools, collaborative analysis platforms, and why all these matter (or should) to security.
• And lest you think that last bullet excludes those organizations unable or unwilling to invest in a major data management undertaking for security: a discussion of how data-driven security will benefit “the rest of us.”

Added to this greater depth are findings from a survey we conducted among 200 respondents worldwide representing organizations of 1,000 personnel or more (since we felt that larger organizations were more likely to invest in the trend), with a mix of results both expected and – in some cases – surprising:

• When asked about the amount of security data they collect, nearly half (40%) of all respondents say they are already overwhelmed. Of the 49% of respondents reporting knowledge of security log and event data collection, 13% report collecting a terabyte or more each day. Some security product vendors, meanwhile, deal with more than 5 terabaytes of new threat data daily.
• At the same time, however, a whopping 73% of survey respondents said they would collect more security-related data, or a wider variety of data, if they could make use of it.

Why this seeming paradox?

• Only half (52%) of surveyed organizations claim to be “somewhat” confident they could detect an important security issue before it has an impact. Twelve percent are neither confident nor doubtful, while 7% report greater doubt than confidence.
• “Too difficult to distinguish legitimate from malicious activity” is the most frequently mentioned frustration with security practices (33% of all respondents).
• 28% say they are about as unsuccessful as they are successful at correlating security data to business impact. 4% are more unsuccessful than successful.

One of the more surprising findings was the extent to which organizations are exploring advanced approaches to data management in support of security efforts. I don’t mean just those normally associated with infosec, such as Security Information and Event Management (SIEM).  Many seem to have concluded – erroneously – that the intersections of Big Data and security are primarily about SIEM.  While it is true that 79% of those collecting security-relevant log and event data in our survey use SIEM and related technologies, the story certainly doesn’t end there – far from it. In fact, it even goes beyond more recent growth areas for security data, such as investigative and forensic platforms that enable security analysts to see threats more clearly:

• Half of all respondents indicated that data warehouse systems are already in use in support of security data management. 38% report the use of integrated analytical database platforms for this purpose.
• 55% employ Business Intelligence (BI) tools in connection with security efforts. 43% use, or have used, tools such as R for more advanced or flexible security data analysis.
• 28% plan to evaluate NoSQL platforms or environments such as Hadoop for security data management.
• Over one-third are already expanding their investment in both technologies and expertise specifically to increase capability in security data management and analysis, with over 40% more planning to do so in the next 1-3 years.

These are just a few of the findings that reflect an interest in data-driven security in the enterprise that may be more active than many may suspect. Clearly, this data will raise some questions. For me, the more tantalizing include:

• What is the extent to which respondents are actually “using” or exploring these technologies? Are these proof-of-concept pilots, or are they actually being deployed in production?
• Are these platforms and tools “owned” outright by security, or are security teams taking advantage of resources made available by other efforts within an organization, such as a BI initiative?
• How mature are these efforts, if in operation – and how have organizations been able to build the necessary maturity to make the most of the opportunity?
• And perhaps the most important: It’s interesting to learn that organizations are using advanced data management and analysis platforms – but what are they doing with all this data? What are the analytic techniques that matter? And what issues are organizations encountering in the responsible management of information that is likely sensitive?

My expectation would be that the answers to any of these questions today would say that most enterprises are still early in the evolution of data-driven security. Not so, however, for some technology vendors and service providers who are among the pioneers of data-driven approaches:

• The ability to mine enormous volumes of threat data to reveal new attacks and help pinpoint compromise have already been embraced by security technology vendors to help organizations deal with the scale of the threat landscape (or landscapes, considering the variety and diversity of threats).
• Putting the fruits of analysis directly to work in countermeasures that have a more direct and dynamic reliance on data sources is one way security technologies may be transformed by the trend.
• Cloud-based approaches offer a means to deliver this capability across a wide audience.
• Providers of managed and professional services, meanwhile, offer the expertise organizations will need to develop their strengths and capabilities in security data management and analysis.

We explore a number of examples of these developments in the report.

I hope to have the opportunity to follow up this research with further insight into the trend as it continues to unfold, and even more examples of the ways in which data science, which is influencing so much of technology development, is transforming the nature of security as well.

So stay tuned! More to come on this aspect of security evolution!

First and foremost:

Data-driven security isn’t just about SIEM, security operations, or “Big Data”:

• Data need not be “big” to have a positive impact on security. (And “big” doesn’t always mean volume alone.)
• Data-driven techniques are influencing a lot more than security information and event management platforms for the enterprise. (Catch up on my overview of the trend as a whole here.)
• “Operations” doesn’t just mean the SOC. The goal of security automation, for example, may well depend on better integrating security data into IT operations management at scale. (If you don’t believe me, ask former Orbitz CISO Ed Bellis, now CEO of HoneyApps.)
• And if it’s true that security ops teams are already too overwhelmed to take advantage of evolving data-driven approaches, how do they expect to get a better handle on where their priorities would best be focused? Or are we condemned to perpetual water-treading (when we’re not sinking outright)? If not, what is the way forward?

In recent days, no one has put forward an answer to that last question more forcefully and convincingly than Ben Sapiro, in this post at Liquidmatrix. By identifying how and where a data-driven approach is central to, as Ben puts it, “build[ing] powerful new techniques and practices for our profession,” this voice adds to those I’ve been following over the last several months that highlight how evidence and insight may help us do what compliance, fear or guesswork has failed to do for us up to now.

So we’re not there yet. Then isn’t it high time we started on the journey?

This, then, is the opportunity I see at this year’s RSA, in spite of the noise. In many respects, this is a year of early exploration and pioneering experience that won’t come again in infosec, so no better time than the present. As Ben describes in the Liquidmatrix post, “I think we are, as [Dan] Geer so eloquently puts it, ‘… at a cross roads, an inflection point’ and now is a great time to build…”

Keep in mind that:

• For as much as data-driven techniques are changing the nature of security data management within the enterprise, they are also are influencing the evolution of the tactics of defense that will find their way into a much larger breadth of organizations. (Indeed, considering the volume of information vendors and service providers must manage across multiple customers, they may be the greatest beneficiaries of the trend.) RSA attendees should therefore look for these distinctions in conference sessions and vendor pitches.
• When data management within the enterprise is a priority, organizations should evaluate where they could do better with mature technologies where expertise and support are well established.
• Regardless, many will have wrung as much out of their legacy techniques as they can. Those considering new alternatives for massively parallel data processing should arm themselves with information about where to turn next. In particular, they should be clear about their objectives, the nature of emerging technologies that can help, the requirements they’ll be taking on if they consider new platforms for security data management, and suppliers who recognize the limitations of emerging tools and can provide an enterprise-class approach with better support than a “science project.” (There will be a session in which an enterprise that has succeeded with Hadoop for security data warehousing will be sharing their experience – see below.)

Your organization may not yet be ready to make any new commitments in any of these areas. In many cases, that’s more than justified, particularly considering the relative immaturity of some emerging technologies, or the impact significant changes would have on existing operations.

That’s just fine. Go and learn regardless, so that you can discover where – or if – a data-driven approach may best (or better) help you. Bear in mind that some of your preferred suppliers may already be moving in this direction with the products and services you already use. If you’re a customer of cloud-based or managed security services that often deliver new capabilities with high transparence, you may benefit in some cases without additional investment or effort on your part.

Here are some sessions that may help:

• I am personally looking forward very much to a panel I’ll be moderating on Monday at 3:30 PM at Mini Metricon 6.5, “Data Mining Methods for Enterprise-Level Security.” We’ve assembled a diverse group representing enterprises, vendors and service providers with a wide range of interests, data sources, and early experience. The Metricon community as a whole has been a pioneer of this trend, and I expect to see more than a few attendees who I know are advancing data-centric techniques.

• I’ll also be participating on a panel on “Managing Advanced Security Problems Using Big Data Analytics” at 8 AM Thursday (Session SPO1-301), where we’ll be discussing what’s different about such an approach (hint: the technologies of massively parallel data processing aren’t just about huge data repositories. They are a collection of tools that continues to expand – and as with any emerging technology, they come with their own set of security issues that vendors are beginning to address). We’ll also consider when and where such techniques may be warranted, and whether enterprises can benefit as well as vendors and service providers that manage data at speed and scale.

• If you’re interested in learning how such an approach is being applied today (yes, today) in the enterprise, the Zions Bancorporation team will be following us on Thursday to talk about their experience in security data warehousing using Hadoop “for long term large-scale security, fraud and forensic related analytics”: Session TECH-303, “Security Data Deluge—Zions Bank’s Hadoop Based Security Data Warehouse,” Thursday at 10:40 AM.

• For a look at how Big Data techniques are changing the nature of security products and services, check out Session DAS-108: “Big Data and Security: The Rules Have Changed,” Tuesday, 3:50-5:00, where a vendor (Sourcefire) and a service provider (Perimeter e-Security) will discuss how these trends are influencing their strategies.
  

• Be looking for the impact of data-driven approaches at Innovation Sandbox, 1-6 PM on Monday afternoon.

• Tuesday at Noon: Special Forum on the Future of Cyber Security and Active Defense: Check out the abstract: “Active defense…emphasizes real-time information, broader situational awareness, and speed…Compare this kind of system to the widely used and disaggregated, enterprise-level approach.”

• Wednesday, 9:30-10:20 AM: Session GRC-202: “Adversary ROI: Why spend $40B Developing It, When You Can Steal It for $1M?” Don’t let the catchy title distract from the focus of this session on shifting attention to better understanding why adversaries focus on what they do, and where organizations may be well advised adjust their strategies accordingly. (For a similar, data-centric analysis, see this article that appeared in the November/December 2011 issue of IEEE Security and Privacy.)

• Also Wednesday at 9:30: Session SECT-202: “CXO Perspective on Addressing Cyber Threats and Opportunities”: One of the topics to be addressed include “information sharing (obstacles, liability concerns)”.

• Sessions devoted to monitoring topics include “Optimizing Security for Situational Awareness” (SPO1-106, Tuesday at 1:10), “Stop the Maelstrom: Using Endpoint Sensor Data in a SIEM to Isolate Threats” (TECH-107, Tuesday at 2:40) and “From the Bottom to the Top: The Evolution of Application Monitoring” (SPO1-202, Wednesday at 9:30)

And don’t forget the non-cons surrounding RSA. In addition to Mini Metricon, BSidesSF will also feature an informative and always engaging lineup of speakers, including some who may not be so in-your-face about data-driven techniques as you may see across the street, but whose experience will likely benefit anyone seeking to learn more about this trend.

Some BSides talks along this line:

• “Hacking the Bank: Figuring out what the cost of hacks may be” (Gillis Jones, Monday 2-3 PM)
• “Building your Own Zombie Horde: Dynamic Web Scanning at Massive Scale” (Erik Peterson, Monday 3-4 PM)
• “Metrics that Don’t Suck: A New Way to Measure Security Effectiveness” (Dr. Mike Lloyd, Monday 4-5 PM)
• “The End of Security Stupidity” (Amit Yoran, Kevin Mandia, Ron Gula and Rolan Cloutier, Tuesday 11-12)

And a few BSides talks where a data- or intelligence-driven approach may not be front-and-center, but which seem likely to provoke your thinking regardless:

• “Playing to Win: Designing Protection Based on Mob Rules” (Matt Stern, Monday 10-11 AM. I would expect more goodness on a practical approach to adversarial thinking.)
• “We are Handling Security the Wrong Way” (Brett Hardin, 1-2 PM Monday)
• “Fundamental Flaws in Security Thinking” (Martin McKeay, Tuesday 1-2 PM)
• “I can read your mind” (Will Tarkington, Tuesday 4-5 PM)