The following is an edited excerpt from Application Performance Management (APM) in the Digital Enterprise: Managing Applications for Cloud, Mobile, IT, and eBusiness by Rick Sturm (CEO, Enterprise Management Associates), Carol Pollard (bio link at: https://cis.appstate.edu/faculty-staff/carol-pollard-phd-0 ) and Julie Craig (Research Director for Applications, Enterprise Management Associates). The book is available now from Amazon.
APM in the Digital Enterprise was published in March 2017 by Morgan Kaufmann, an imprint of Elsevier. The content covers the gamut of application management-related topics starting with the evolution of APM, to DevOps and Continuous Delivery, APIs and connected systems, User Experience Management, and Distributed/Componentized Applications (see the full table of contents here).
The book combines the knowledge of all three authors, each of whom has worked in the IT industry for 30 years or more. It is well worth a read for IT professionals involved in any stage of application delivery across the lifecycle, IT executives tasked with overseeing application delivery–related activities, and front-line personnel—developers, DevOps professionals, and operations teams—responsible for any aspect of application delivery. Members of the press and others who need to understand APM will also find the book a valuable resource.
This article condenses some of the key concepts covered in Chapter 11, entitled “Application Programming Interfaces and Connected Systems.”
“Today, everything is connected to everything.”
—IT manager at a global bank
We live in a world of massively interconnected applications and supply chains. In recent years, the use of Application Programming Interfaces (APIs) has largely replaced technologies such as Electronic Data Interchange (EDI) and custom-written programs for development of new system integrations. APIs are now the de facto industry standard for integrating data and/or functionality across diverse application ecosystems.
The growth of public and hybrid cloud, mobile devices, containers, microservices, and Internet of Things (IoT) has accelerated the need for application and data integrations. Industry standards such as REST and SOAP have facilitated the process. APIs built over these protocols simplify, and, to some degree, standardize the integration process. They reduce the need for the bespoke integrations of the past—which were required to support exotic protocols and proprietary operational systems. In short, APIs have become the standard currency of exchange connecting applications, devices, and companies.
API Providers vs. API Consumers
There are two sides to the API coin: “providing” and “consuming.” Growing numbers of companies are consuming APIs to access data and functionality exposed by other entities. And a large number of companies are acting as API providers, exposing their own systems to those of customers, partners, and suppliers. Many companies are doing both, and some are monetizing access to data or internal systems as part of revenue generation.
The speed and breadth with which API ecosystems have proliferated is impacting APM in a big way. Applications relying on APIs to provide data or functions necessary to complete a transaction—an internet sale, for example—can be slowed or stalled by many of the same factors that can impact other tiered, distributed transactions. At the same time, however, APIs leverage new protocols, connection methodologies, and architectures that may not be supported by traditional APM products and methodologies.
In short, while APIs are the new standard of B2B and B2C interchange, they also introduce new management challenges that many companies are not equipped to address. Usage growth, for example, can be a major problem that can significantly impact performance. In July 2015, EMA published a report called “Back to the Future with the API Economy: Management Strategies for a New Wave of Integrated Applications.” While the study covered both API consumer and API provider use cases, an examination of the issues facing API providers was particularly interesting.
The top three challenges identified by respondents from companies providing APIs included:
- High traffic volumes
- Security of back-end systems
- Identity and authentication management
As an example of issues relating to high traffic volumes, API providers most commonly indicated between 500,000 and 1 million transactions per month accessing their APIs. However, more than 50% reported 1 million or more transactions per month with a small fraction – 3%– reporting 1 billion or more. In addition, 85% indicated that transaction volumes were increasing, most often between 10% and 20% per month. This massive growth can tax the resources of existing delivery systems. To make matters worse, many IT organizations do not, as yet, routinely take API-delivered services into account when doing capacity planning.
Participating in the API Economy doesn’t stop with providing or consuming APIs. Security, access, metering, chargeback, and other API-related functions also become increasingly relevant as usage increases. And as the number of API provider and/or API consumer connections grows, as more users and applications connect, and as new API versions are created and deployed, the API Economy begins to look more like a maze to be navigated than a straightforward way to flexibly extend organizational borders.
API Management Tools
So how and where do tools fit into this picture? Tools help rein in this growth and complexity by addressing key functional questions supporting tools acquisitions.
API providers often find themselves asking:
- How can we track usage growth and the impact of that growth on back-end systems for capacity planning purposes?
- How do we ensure that only authorized users and applications connect to our systems?
- How can our organization synchronize API development with traditional application development lifecycles since the two are often linked?
- How can we secure API usage to ensure that sensitive data is protected?
- How can we track usage of “for pay” services to correctly bill for access?
API consumers ask:
- How do we find out about new APIs offered by our vendors and partners, and how do we then go about accessing them?
- How do we know when the APIs our systems are accessing are modified by the provider?
- We have hundreds of applications that access APIs—and some of them interact with one another. How do we measure end-to-end performance? And when one such application fails, how can we determine what changed, what’s wrong, and how to fix it?
A large majority of both consumers and providers are monitoring performance and availability of applications accessing APIs from the perspective of the gateway. Although this is a good starting point, it is essentially a silo solution to a far broader and more complex problem.
API-connected applications, like any other tiered, distributed, or componentized applications, may have hundreds or thousands of potential failure points. From this perspective, simply monitoring the gateway is akin to monitoring server, network, or database silos. Such monitoring fails to address the touch points BETWEEN hardware and software elements that occur during application execution. In other words, it lacks the visibility to the entire end-to-end execution path that distinguishes application management from systems or silo management.
In the end, APM platforms—and API management systems– should have mechanisms for incorporating gateway performance data into analytics, correlations, and dashboards. A few vendors are already addressing the API market with data-sharing capabilities and/or and products specifically designed to manage API-connected applications across each stage of the lifecycle. Lacking a single point of visibility and control to application execution (versus silo performance), full automation of the end-to-end monitoring/management function remains a fruitless quest.
EMA is currently in the process of launching new research into the automation and tools supporting the API Economy. This study, “Enterprise Management Strategies for the Connected Business: Hybrid Services and API Ecosystems Become Business as Usual”, will include both a user-facing survey and vendor “snapshots” encapsulating the types of tools and capabilities currently available in the enterprise management tools market. For ongoing updates on this research, visit www.enterprisemanagement.com or contact EMA sales at: firstname.lastname@example.org.
 Available for download at: http://www.enterprisemanagement.com/research/asset.php/3059/Back-to-the-Future-with-the-%22API-Economy%22:-Management-Strategies-for-a-New-Wave-of-Integrated-Applications-