The EMCWorld 2013 event started first, kicking off with the news that EMC had built, trialed, and released a new virtualization layer called ViPR (pronounced “viper”).  The name does actually stand for something, but it’s so abstract that it’s not worthy of mention (plus I was sworn to secrecy).  Viper sounds cool, so ViPR it is. But seriously – this is not just a rebranding of storage virtualization.  It is a true software-defined architecture, with a services layer, a controller layer, northbound APIs, and a promise of true multi-vendor support on the southbound side.  Now let’s be honest here – this is an EMC-centric story – there was nary a whisper on standardization, though there were fervent promises of multi-vendor support.  But storage does indeed need to come along to the programmability party if VMware’s grander vision of the Software Defined Data Center (SDDC) is every truly going to take flight.  For EMC’s part, they did it right – built it out, found early adopter customers (CSC, UBS) and saw it through into production deployments.  So the solution is real, not just vaporware (or should that be ViPRware??).

A little off to the side of this big event was a parallel release by the Infrastructure Management team that will be of interest to network and infrastructure managers.  This latest Software Assurance Suite release included a new consolidated GUI built using the thin, HTML5 frameworks originating from acquiree Watch4net.  There’s also a very cool new Smarts feature called Watchlist, which consolidates availability, performance, and configuration management data to present health, notifications, change history, compliance status, and impact for devices, groups of elements – even service groupings.  This is great progress towards business-aligned operations, and is a clear value-add for anyone using the EMC management platform.

I was asked to host a BoF session on Converged Operations Management as part of the conference program.  Not surprisingly, the largest contingent was comprised of storage managers, but there were also many virtual systems managers. Network managers were scarce.  While a significant number of attendees indicated that they were trying to move towards converged, cross-domain operations, two key frustrations quickly emerged in the conversation.  First, many expressed serious challenges finding qualified cross-domain generalists to staff the function.  The two most practical answers were to hire junior folks that had training in one technology domain and put them through cross-domain OJT, and to look for folks that have had to wear multiple hats (by necessity) while working in smaller IT shops.  The second challenge was that there didn’t seem to be any consensus on what tools could deliver the operational views necessary for supporting a converged team.  Some had built their own, many were focused on vCenter Operations, and some were using a mishmash of disconnected tools.  Clearly we can do better, as an industry, and make sure that management tools are not a barrier to this important evolutionary progression.

As always, the time spent talking with management product experts and real-world practitioners is invaluable in keeping my own research well grounded.  But next year, if these two events happen in parallel, maybe I’ll hire a telepresence robot, like the ones you see being used for telemedicine or remote learning.  Maybe cloning will have been perfected.   A Harry Potter-esque time turner could do the trick.  But most likely, I’ll just wear out another pair of shoes.

First up, I talked to the CA AppLogic team.  Now AppLogic is not where you might expect to find network management, as it is presented externally as a cloud resource manager that virtualizes IT infrastructure in the service of flexibility and agility.  For most, this means focusing on compute resources along with direct-attached storage, but AppLogic also recognizes and addresses some essentials regarding network path connectivity.  As it turns out, AppLogic does its own network resource discovery, and builds/maintains an internal model of these resources as part of its virtual resource pool.  AppLogic then apportions connectivity, including expected network bandwidth and prioritization needs, as part of deploying and administering work loads via what amounts to a virtual overlay network.  AppLogic does not touch any network devices to do this – it just directs and manages the capacity it finds and believes is available across the network.

I also spent time digging into the CA APM story.  There is an interesting technological parallel that I’ve been tracking between the use of packet capture and analysis within app-aware network performance management (ANPM) tools and also within traditional application performance management (APM).  See EMA’s ANPM Radar Report 2013 for more on that, but suffice to say that a number of management tools vendors are bringing together APM and ANPM techniques in addressing market needs for application performance visibility.  CA offers the Customer Experience Manager, a component of the CA APM (nee Wily) solution that is designed to measure customer experiences by analyzing packet streams going into and out of a front-end web server.  CA recently expanded the scope of its APM monitoring and diagnostics by integrating the CA Application Delivery Analysis (ADA) appliance.   The ADA is deployed at various points around the network, commonly between tiers in the data center, at datacenter ingress/egress, and even at remote sites, to provide a more comprehensive view of application traffic flow.  Now interestingly, while CEM has Wily heritage, ADA began its life as the NetQoS SuperAgent product, which was squarely aimed not at application support but rather at network engineers (a la ANPM).  So this is a case where network management technology has been dual-purposed to support APM directly.

Finally, I spent some time getting an update on the CA Service Operations Insight (SOI) product.  SOI is an operations bridge that takes inputs from element and domain managers, relates them via service modeling, and displays health information on a service-by-service basis.  Connectors are also provided for CMDB and ServiceDesk.  SOI includes pre-verified plug-in adapters for Spectrum and Performance Center, as well as the APM suite (and much more).  What is interesting here is that while many technology vendors have tried to build service awareness into their network management tools, CA has taken a different route, applying service awareness at the operations bridge layer.  I talked to three CA customers who were using this combination, in all cases bringing CA Spectrum and CA APM data into SOI, and all said it was both easy to deploy and really helpful as a means for elevating each data silo into a common, integrated view.  Again in all three cases, the result has eliminated finger pointing among traditionally siloed teams when things go sideways.

Net net, this is all good for the future of network management.  These points of integration bring the network viewpoint directly into its predestined role – a critical component of truly systemic, service-oriented planning and operations.

1. Find your place in the clouds. The use of cloud services continues to grow, and it’s still not clear what role network managers have in this fundamental shift for IT sourcing strategies, beyond reliable connectivity and (sometimes) optimized performance. Most important to understand is that internal IT organizations are also shifting, slowly but surely, towards an internal private cloud service provider model. This has significant implications for network managers in two ways. First, cross-domain operations teams, which are focused on application performance and end-user quality of experience, are becoming increasingly common, and to be successful must include the network manager’s voice and participation. Second, all network management tools, technologies, and practices must be reevaluated and aligned with application/service objectives.
2. Take a SysAdmin to lunch. Whether you have realized it or not, new network elements are popping up in your environment whenever virtual server hosts are deployed. Add to that the ongoing “conspiracy” that virtualization vendors are fomenting via virtual overlay networks to do an end run around real or perceived network configuration and change barriers. Those vendors have one goal – to give virtual systems administrators all the tools they need to spin up and move VMs at their every whim. We all know that the old-fashioned network plays an essential role here, so it’s best to find out what the system administrators are thinking and how seriously they are planning on new network virtualization solutions from non-network vendors.  So take a chance and ask a SysAdmin to lunch, and find out what they are working on –it’s a neighborly thing to do and it might be an eye-opener to boot.
3. Integrated management is back. The pendulum is swinging away from best-of-breed, multi-product management strategies towards fewer, more tightly integrated management platforms, both at the network management layer and elsewhere in the management stack. This is driven in part by the organizational shifts mentioned above but also by real potential savings in total cost of ownership and operations, as well as real improvements in operations awareness and responsiveness. Further, such integrated approaches are proving effective for those who want to be more proactive. Take a look at your network management tools and asked the question, “Will they allow me to reach a more integrated, cross-functional, proactive level of operations?”
4. Do your homework on SDN. For most enterprises, Software Defined Networks (SDN) are still a curiosity, but the technologies are finding rapid purchase in the service provider and web-scale commercial sectors. It’s too early to say when, exactly, SDN will be ready for mainstream enterprise networks, but 2013 is a great time to start researching and understanding them. EMA has been tracking this trend (see blog posts here and here) and is planning to conduct direct research into this area early in 2013. Specifically, we will focusing on manageability – including network monitoring, troubleshooting, and security – and what is needed for them to be ready for the broader enterprise community.
5. It’s all about the Applications. Like it or not, the network is only top-of-mind with IT end-users when there is a problem. The touch point between human and network is facilitated most directly by the applications and services that they use for their work and daily lives. The network’s essential role is to deliver those applications and services reliably and with maximum performance. So everything that a network manager does should always be informed by or influenced by the way in which the network fulfills that prime objective. If your current network management tools are not providing you with application awareness, this is the year to change that. Start by looking at NetFlow or IPFIX records, or by adding selective packet inspection instrumentation to get such visibility.

What are you planning to do this year? How will you make your day-to-day work more effective and how will you plan for the future? Please comment with your own resolutions!

My colleague Tracy Corbo and I have been spending a lot of time talking to vendors and studying the SDN hype-trend, in order to try and peel back the hyperbole to reveal the essential truths that mainstream enterprises will need to understand. Along the way, we have formulated a couple of key foundational conclusions:

1. The SDN universe can be roughly divided into two categories:  Virtual/soft overlay networks (think VMware/Nicira and BigSwitch) and the “let’s commoditize the delivery plane” group (think OpenFlow and ONF).  Tracy wrote a great post on this that goes into more detail.
2. To date, virtually all real-world deployments of SDN have been in academia, service providers, or really huge on-line commercial entities (a la eBay, Google, Amazon, Facebook).  While interest is growing within mainstream enterprise, it is still just interest, in part because the enabling technologies, while really exciting, require small legions of programmers and staff to deploy and maintain.
3. Across the board, significant hurdles remain in terms of stable, reliable, deterministic management and monitoring. In fact, we are marveling over the stunning lack of progress on the management side of SDN.

In many ways, management “lag” is not surprising. Management is often an afterthought when a new networking technology (or most any IT innovation) comes about. However, it is not a question of if, but rather a question of when management best practices must emerge if SDN is ever going to be ready for use by the mainstream.

One big question is whether or not SDN–based networks can be addressed using traditional network and security management technologies.  On the plus side, most of the underlying architectural concepts are really not all that different from others that have preceded SDN, and so it should be possible to adapt existing systems. On the downside, complexity and dynamicism are going to increase radically, and not all existing management tools will be able to accommodate the scale and rate of change that is coming.

For overlay network types of SDN, we simply need to adapt existing tools for recognizing the new virtual network protocols and constructs, instrumenting them to measure performance and availability, and extend configuration and provisioning tools to support them.  The big challenge here is defining, discovering, and recognizing the relationships and touch points between the virtual overlay and the underlying true physical network.  This knowledge is crucial for responsible planning as well as for any hope of effective monitoring and troubleshooting.  Orchestration will play a huge role, as will advanced performance analytics and root cause analysis.

For OpenFlow types of SDN, there are separate sets of challenges.  Many of the basic physical managed components will be the same as before SDN, except that we now must also understand the critical role of the Controller. Further, there is an entirely new technology – OpenFlow signaling – that must be included and accounted for across the full lifecycle of planning, deployment, monitoring, troubleshooting, maintenance, etc. Because OpenFlow is control traffic, it can be a source of useful operational intelligence, but it must also be closely monitored as a new breakpoint and threat point in the architecture.

Further, there are two categories of control traffic to understand and accommodate. The first category is requests that will be coming into an SDN controller from some number of sources that are seeking network services. The second category is the commands that are subsequently sent out from the controller to the networking layer. Some big questions still exist with these layers. How do we assure that the commands are real/valid? How do we deal with resource contention? How do we arbitrate between competing or overlapping requests or commands? Much of this will be solved within the controller architectures, but these are precisely the sorts of issues that may be the root cause of poor performance or non-performance at the delivery layer.

What we are facing is the need to adapt prior art in terms of network and security management technologies and practices and bring them into the mainstream of enterprise planning and operations. This is what we at EMA are calling Responsible SDN – deploying SDN with full confidence that it is production ready, enterprise grade, secure, and manageable.

Tracy and I, along with Scott Crawford, who leads EMA’s security management practice area, will be looking into the Responsible SDN topic much more deeply in enterprise-facing research due to start in January 2013. If you have ideas or opinions about this topic, we would love to hear them!

I’ve just returned from Interop in New York, and it seemed like every other sentence spoken there included “SDN.” The networking community is abuzz with talk about Software Defined Networks (SDN), but what does this really mean, in terms of what we should expect out this latest, most-hyped term? Current views (and agendas) are fragmented and often too narrowly focused. Many see it as a battle cry for commoditizing network delivery and the end of proprietary network equipment dynasties. Others see it as simply applying the same principals that made server virtualization successful to networking infrastructure and thereby removing the last barrier to the supposed nirvana of the “software defined data center.” In EMA’s view, both of these angles are not only wrong, but are creating a huge set of misconceptions about what SDN really is and is not.  The power of SDN has nothing to do with software, the controller, or the data plane – it is about services. A better definition for SDN would be “Service Defined Networks”. EMA believes this is a much better way to describe the true transformative potential of the initiative as a whole. True SDN represents the long–awaited opportunity for networking to be directly connected to the demands of the business, as supported and addressed within the application tier.

EMA has long been an advocate for application awareness within the network management stack, as a means for monitoring and assuring that the network infrastructure plays an essential role in delivering applications and services. With such an approach, any disruptions that occur can be recognized and mitigated in a manner consistent with the priorities and business value of each of the individual applications and services. This concept can and should be extended into the planning and configuration process as well, where business and application priorities are included in the planning process.  For example, consider the planning required to support a VoIP or VDI rollout, both of which demand special care and feeding from within the network layer.

But things change, and at an every increasing rate.  Connecting the constantly changing world of applications and services to the supporting infrastructure in a way that allows infrastructure operations to be fully in tune and aware of changes in the application layer has been a major roadblock. To date, the vast majority of networking configuration and monitoring changes are made in an ad hoc, manual fashion in response to application moves, adds, and changes. The advent of SDN will require well-defined northbound interfaces to drive end-to-end policy control, representing a means to building automated responses that would enable a proactive exchange between the network and application layers. How this automation is achieved can be done either on the network hardware platform, in the networking software, or some combination thereof, but the end game would finally achieve the delivery of services over an optimized infrastructure driven by application requirements.

So that represents the potential for what SDN if fully realized can be. For now we are caught in a sort of dizzying array of things that claim to be SDN, but are really not. There is more than one way to reach the ideal end point and what we are seeing now are the interim steps to moving in that direction. The path of progress is all about finding ways to make the network architecture more flexible and adaptable and less rigid in design. Let us be clear – networking hardware is not going away, nor is the need for sustained, ongoing innovation within the networking layer. But what is happening is that networking equipment will have to become more application-attuned. If we truly seek to virtualize the network layer, the implementation details within the network layer are less important than what is presented to the rest of the world, which is an automated, service-oriented network delivery service.

We are working towards the concept of northbound APIs into the application layer coupled with consistent, automated policies applied to the networking infrastructure. OpenFlow and general-purpose controllers are all first steps in the direction of realizing SDN, but we are nowhere near the finish line, and much of the story has yet to be written.  EMA will be tracking this important revolution closely throughout 2013 and will be publishing our findings along the way.  Stay tuned – this is going to get very interesting!

Enter upon the stage another latency-sensitive application: VDI. Virtual Desktop Infrastructure is all the rage as a means to reduce security risk around data leakage in the face of inevitable BYOD and endpoint consumerization trends. There are a number of other driving business reasons for embracing VDI, but BYOD is, from my viewpoint, the most compelling potential tipping point.

Now VDI is not everywhere yet, but interest is high. During my recent Network Management Megatrends 2012 research project VDI turned up as the second most-influential broad IT initiative in the minds of IT execs when it comes to networking and network management.  The only higher response was for server virtualization.  Curiously, networking pros were much less concerned – half as much as the execs – which indicates that adoptions are still early and the full effects have perhaps yet to be fully appreciated.

But why should we care about VDI from a network perspective? There are multiple reasons.  First and foremost, separating the human interface, where mammal meets plastic, from the actual instance of a desktop application execution environment, which is now hosted on a remote server, introduces a new performance variable – the network. Secondly, VDI performance relies on network architecture and plans in the back office /datacenter as well. And lastly, hosted desktop services via VDI technologies (a.k.a. Desktop as a Service, or DaaS) is a growing field, and can never be fully assured without some degree of mindfulness about the role and influence of network access.

In terms of delivering an optimal VDI experience between the VDI server and the client, the challenge primarily revolves around network optimization and traffic prioritization.  And very important in this is consideration of exactly where the client sits. Many early VDI deployments have achieved success in part because they are restricted to LAN/campus environments. But the greater challenge is to extend the benefits of VDI to clients that reside remotely, either across a corporate WAN or via broader Internet access. Here, traditional technology solutions such as application delivery controllers (ADCs) and WAN optimization controllers (WOCs) can be applied and can add significant value. Take a peek at the noise that F5, Array, and Silver Peak are making around this.  Another valuable approach is to look at Internet optimization services to improve delivery effectiveness – see Akamai’s Terra Solution if you are looking for examples here.

On the backend, in the data center, there are also plenty of opportunities to improve and optimize the networking aspects of a VDI deployment. In particular, VDI servers host the desktop sessions, but are essentially front-ends for traditional multitier applications. Further, the access adjacency of storage is hugely influential in the total VDI performance picture. Done properly, the end-user experience can be even better than a physical, native desktop. Done improperly, that same end-user will be substantially degraded.  For more on this, take a look at what the VCE Coalition and V3 Systems are doing.

Later this month, I am launching a primary research project into the best practices for planning, monitoring, and optimizing networks to support VDI deployments. In particular, I’ll be looking for clear emerging trends and requirements so we can all learn from those who go before us.  If all goes as planned, I will have results to share by the end of June. So stay tuned! The results could truly be “killer”…

1. Don’t wait for the Cloud to come to you. Cloud services are big emerging topics and goals in the minds of IT execs and IT departments, more broadly, but the networking team is often not involved in these conversations. And yet, experience tells us that when it comes time to put cloud services into production, whether that means entering into service agreements with external cloud providers or remaking IT to be an internal cloud, that networking will absolutely be an important aspect of production operations. What’s the best way to put yourself into position to help? A good start would be making sure you have application-aware monitoring tools, such as packet inspection or NetFlow, ready to go so that you can see how these services are being used and how well the network is playing its role as cloud services ramp up.

2. Use Converged Infrastructure projects as an opportunity to consolidate.  There are number of definitions of converged infrastructure out there, ranging from latest-generation networking fabrics to multi-domain pre-integrated “data center in a box” solutions such as FlexPod and Vblock, which deliver compute, hypervisors, network, storage, and sometimes even pre-installed applications, all in one pre-built unit and arriving on a single skid. If your organization is moving towards that latter category, the opportunity exists to ride that wave and bring truly integrated, cross-domain management tools and practices to bear. Take a look at EMC‘s Unified Infrastructure Manager (UIM) which was developed initially to consolidate provisioning functions for VCE Vblocks but was recently extended for integrated monitoring. This could very well herald things to come–a new approach for tightly integrated management tools and practices.

3.  Beware the virtual desktop. Last year, EMA spent some time researching/analyzing emerging requirements for network managers to accommodate VoIP and live videoconferencing, such as Telepresence. While these two technologies continue to grow in use, and hence impact to the lives of network managers, the next great network-dependent technology is hosted desktop, otherwise known as Virtual Desktop Infrastructure, or VDI. Like the other two, this is not a new technology, however it appears that other macro-level trends such as mobility and BYOD are driving interest in VDI to a level that is unprecedented. The big challenge, of course, is that if you introduce a network link between a thin virtual client and the server that is supplying all of its data and actions, there is a potential for networks to be a barrier to success. There are other barriers to VDI success, by the way, but it makes sense to look into how ready your network is to handle this new type of application service, by reviewing capacity plans and QoS policies.

4.  Look at the world through Application-colored lenses. This is mentioned above (resolution #1), but in case you have not gotten around to it yet, there is no better time than 2012 to put in place network-facing monitoring technologies that can reveal precisely who is using the network and for what purposes. This does not necessarily mean security awareness, although it can, but more specifically which applications and services are traveling across the network and consuming common resources. Look to NetFlow/xFlow and packet probes as the best technologies here, though log file analysis can also be useful.  This viewpoint puts network operators in a position to contribute valuable operational insights when troubleshooting a problem or analyzing trends that need to be accommodated via policy changes or capacity planning. In the eyes of many, the traditional BSM disciplines are all shifting more directly into APM (application performance management) and the result is yet another affirmation that “The Application is King.” To this I say “Long live the King!” Its reign is irrelevant if there is no network to deliver it.

5.  Seek the proactive path. Rarely do I speak to a networking practitioner who does not express some interest in moving out of reactive mode and into position to be more proactive in supporting their organization. Unfortunately, priorities as they are, this has been much more a dream than a reality for the majority. This is a quest, however, that must be continued. In the coming year, take a look at what your management tools offer in terms of proactive or preventative alerts. Try setting them up around a few high-priority, high-visibility resources or applications. If you can catch and prevent a degradation or outage from affecting a mission-critical application (and you tell someone about it), you will be every bit as much of a hero as when you come in to fight the fire after the house is already burning.

What else are you resolving to do this year?  Please comment and let me know – I’d like to keep track of them all so we can see how we did next New Years….

Let’s take the example of telepresence suites. They are real-life Star Trek Enterprise bridge view screens – life-sized, high-def, and audio-perfect. If this technology were to become ubiquitous, it would truly change the way in which humans interact in fundamental ways.  Just think of the opportunities – set up your telepresence system at one end of your dining room so you could include your distant cousins or grandparents at the table.  It’s an intriguing idea that may come true some day for the consumer market, but we are much closer to this reality within the enterprise.  And in both cases, the true revolution is being driven not by high end suites, but by devices that everyone has or will soon have – front facing cameras on smart phones and tablets.

As much as I am enthralled with this technology, I also understand the technical challenges facing those who wish to deploy it.  From an enterprise network planning and operations perspective, as well as from a service quality assurance perspective, the challenge can be summarized in one word: volume. While one telepresence suite can consume multiple gigabits of bandwidth, their relatively small total numbers means planning and segmenting networks around them is workable. But when every endpoint in a large, distributed network may simultaneously be demanding continuous, high priority real-time videoconferencing communications a minimum rates of 100-200 kbps, the challenge gets quickly daunting.

Consider the case of the international investment bank I interviewed as part of my recently-published research report, Videoconferencing Impact on Network Management. The company has deployed over 10,000 high-definition desktop video devices, and expects to double that over the next 12-18 months. If 5% of those endpoints are active, the total aggregate bandwidth impact remains in the comfortable level of 100 Mbps or less. But on a busy day, where 20% or 30% of those endpoints are active at any one time, that footprint can rapidly approach a full one Gbps. And remember, this is no longer a dedicated network – this traffic must travel a shared network that is also carrying critical business applications and data feeds. See how much fun this is going to be?

That same investment bank has been working on this problem for years, and firmly believes they have a plan to make it work. First off, videoconferencing from the desktop is assigned one half of the high-priority network traffic queue (the other half is for VoIP), which is in turn accorded 30% of planned network traffic to and from distributed (WAN-connected) locations. This is a standard configuration for most sites, however adjustments are made depending on the constituent composition of any particular site, either by shifting the mix between video and VoIP or by increasing the high priority queue size. The bank’s Voice & Video team is populated with personnel that mostly came out of the networking organization. Consequently, they not only have a keen understanding of what impact this traffic will have on the network, but they also have maintained a close working relationship with the network team so that monitoring can be shared and adjustments made quickly and smoothly.

One other interesting finding was the experiences shared around what most often causes video conferencing quality to suffer. Our respondents told us that the most common root cause was operator error – not all that surprising, given that videoconferencing systems are still in their early phase of broad adoption. But beyond that, the next most common causes of problems were (in order):

1. Network traffic congestion in the WAN
2. Network traffic congestion in the LAN
3. Network latency in the WAN
4. Network latency in the LAN

These four problems clustered together as more common than a wide range of other potential sources, including device health, network device configuration, endpoint device configuration, network device health, and even videoconferencing system health. Quite clearly, the greatest challenge facing successful deployment of videoconferencing (beyond end-user training) is the integrity of the network. What surprised me was that LAN congestion and latency were called out so closely as root causes to WAN congestion and latency – the latter would naturally be expected to be stress points given bandwidth constraints and inherently higher latency contributions. This means that network planners and operators must pay close attention to QoS policies in the LAN just as much as they would for the WAN.

Network optimization also plays a role here. Another practitioner that I interviewed related a story illustrating this point. His shop had deployed WAN optimization controllers (WOCs) to help with QoS policy enforcement and data compression across the WAN. When those WOCs temporarily lost their QoS policy rules during a software upgrade, the team immediately began seeing videoconferencing quality issues arise, and the issues disappeared as soon as they were able to restore the QoS rules to the WOCs.  The experience was a powerful lesson regarding how much videoconferencing relies on QoS.

There’s more, of course.  One area of emerging focus regarding desktop videoconferencing is the way in which it will work (or not) together with hosted desktop technology, such as VDI. More on that in a future post, but for now, please enjoy your holiday…..

Happy Thanksgiving!

Finally freed from the demands of crunching together data for our just-released ADC/LB radar report, it was a good time to check in on the WAN optimization front and see what has been transpiring over the last couple of months. What I found was an interesting pattern. There are the usual product updates and the bit of conjecture regarding the status of one vendor’s source funding, but there is also a prevalent theme and, I am afraid, I must say that I think the connection has to do with that so-overused term “cloud.”  First, here are the items that caught my eye (by no means an exhaustive list):

• September 27, 2011 – NetEx unveiled HyperIP Version 6.0, featuring HyperIP for Hyper-V, a fully-integrated virtual appliance that supports the Microsoft Windows Server 2008 platform.

• September 28 – Blue Coat Systems announced that the Blue Coat WebPulse collaborative defense proactively protected its 75 million users from the latest attack launched by Shnakule, the largest malware network (malnet) on the Internet.

• October 5  – Cisco announced a family of new WAN optimization appliances targeted at the branch. The new Cisco WAAS appliances 294, 594, and 694 appliances enable branch offices to deploy up to eight virtual services such as video, virtual desktop infrastructure (VDI) and Windows on WAAS.

• October 12 – Cisco and Citrix Systems announced that they have entered into a strategic alliance to develop and deliver high-definition virtual desktops and applications and improve end-user experiences over a highly secure Citrix HDX-enabled Cisco networks, including specific new optimization functionality in Cisco WAAS products.

• October 13 – Ipanema Technologies announced the availability of the nano|engine family of low cost, ultracompact devices tailored for branch offices.

• October 14 – Citrix announced the addition of Citrix Branch Repeater (WAN optimization) as a virtual service on the NetScaler SDX application delivery controller (ADC) platform.
• October 18 – Expand Networks – According to an article in an Israeli-based newspaper Globes, an Israeli court appointed a receiver for the company after one of their primary investors, Plenus Venture Lending Fund petitioned the court. While Expand Networks appears to be operating as normal, it does raise concerns over the company’s long-term viability.

• October 18  — Silver Peak Systems announced the availability of Silver Peak VX virtual WAN optimization appliances on HP 5400zl and 8200zl switches.
• October 25 – Riverbed Technology announces delivery of products acquired as Zeus (ADC) and Aptimize (web content optimization) as a new family of Stingray soft appliances.  The acquisitions were announced concurrently back on July 19th.

• November 3 – Silver Peak Systems announces that their virtual WAN Opt solution, VX-Xpress, has received 3500 downloads in its first two months of availability.

As I was looking at all these various announcements, a couple of things jumped out at me. One this is a tough market. As the market has reached a level of maturity, it is not just enough to push applications faster through fixed pipes. Many things have changed since these appliances first hit the market. The initial goal was to improve application performance between the datacenter and the branch offices. Now with the growth of mobile devices, that demand is moving out of the branch and onto wireless and carrier networks. This presents problems both from a performance and a security perspective. The ability of end users to introduce malware and viruses into the corporate network will continue to increase.  How do companies keep their networks secure?

Next: Cloud, like it or not, is here to stay and it too is changing how networks are architected. It demands asymmetric deployment models and virtualization is a must. If you want to play in the cloud you have to be able to deploy in this fashion.

The rate of change has accelerated, making it very difficult to keep up. Change has become a constant. These announcements not only reflect the need to remain relevant in this fast paced market, but it also speaks to how things like mobility and cloud are driving not just IT departments to work together, but IT vendors as well. Citrix and Cisco each have their own WAN optimization technology. However, in some cases based on customer demand, it is better to partner and provide a single point of contact rather than create a point of contention. For vendors such as HP it makes more sense to work with an existing WAN optimization vendor rather than trying to build their own solutions, because the required time to market is too short and the pain point is too high to do otherwise.

Vendors in the WAN optimization space need to remain relevant and at the same time find ways to differentiate themselves among all the other options out in the market. All this is happening during a time of decreased spending and more competition for every IT dollar, so it make sense that rivals might want to find ways to work together so that they can both get a piece of that ever-shrinking IT pie.

Overall, the WAN Optimization sector promises to continue changing and being an interesting area of technology development.  The value delivered is still compelling, and demand is strong enough to keep driving innovation and competition.  EMA will keep an eye on the developments and do our best to keep you informed!

This should go a long way in helping with one of the major issues that network engineers and managers have with virtual server environments – the loss of visibility into traffic that is going between virtual machines on the same host.  I’ve written on this before, referring to it as “the fog of virtualization.” This blind spot has significantly hindered both understanding performance and troubleshooting of degradations. In an EMA research report that earlier this year, network managers called out performance and visibility as critical needs for managing in and around a virtual server infrastructure.

Application-aware network performance management vendors have been trying to deal with this in a variety of ways. Those who support NetFlow have been picking up and offering reports against the experimental NetFlow records coming out of the older versions of VMware’s vSwitches, but the jury has been mixed on whether or not that data is accurate enough to rely upon. Packet-based monitoring vendors have offered virtual probes and virtual taps that reside as VM appliances and rely upon promiscuous mode connections to the vSwitch. This is more accurate than the experimental NetFlow, but does require sysadmins to agree to deploying tools on their hypervisors that can pull significant CPU cycles. One vendor offering a virtual probe appliance even went so far as to recommend a dedicated CPU core be assigned in order to assure full functionality.

These prior options have been good enough for some people, and many have experimented with them, but they have fallen short of mainstream status in part due to the reasons mentioned above. When I talk to networking pros, many of them tell me they have been pinning their hopes on third-party virtual switch technology, in particular Cisco’s Nexus 1000V, as the best opportunity to solve this issue. The Nexus 1000V offers traditional switch monitoring capabilities such as SPAN and NetFlow. The biggest drawback here has been cost – the 1000V is not bargain priced. Still, at Cisco Live! in Las Vegas two months ago, I was pleased to hear most people saying that they had deployed the Nexus 1000V in production, whereas the year prior no one had moved it out of their test labs.

One other innovative option is offered by Net Optics, with their Phantom Virtual Tap (and just last week the newly announced Phantom HD). Net Optics has taken the approach of tying directly into the hypervisor kernel rather than pulling data out of the vSwitch and into a VM. This has advantages in terms of reducing load on the switching function, but may not be palatable to those concerned about optimizing overall hypervisor/system performance. Still, Net Optics raises a good counterpoint to the vDS SPAN feature, pointing out that it may also represent a significant processing load and loss of bandwidth which in itself may be deemed unacceptable.

On the NetFlow front, the new vDS supports NetFlow version 5 format. That means data from this viewpoint can be integrated into virtually every commercial or open source NetFlow collection/analysis tool. This will be most helpful in providing basic flow monitoring and a quick understanding of the aggregate activity between the VMs on each host. Additionally, NetFlow is commonly used to track activity for security purposes, and this will help with recognizing unusual activity between VMs. But NetFlow v5 does suffer some limitations in terms of its ability to be used for definitive troubleshooting. It will be interesting to see if VMware continues to push forward towards more current versions of NetFlow, such as v9 and IPFIX, which can include much more detailed flow data through flexible templates.

Overall, I’m very excited by the progress being made here.  The new vDS features just may represent *the* visibility answer for the broader masses. While many will still look to the Nexus 1000 V or virtual taps and probes, the vDS now becomes a standard part of the VMware architecture. And consequently, traditional mainstream monitoring will now be possible out of the box.

Finally, perhaps, the fog of virtualization is lifting…..