Application Auto-Scaling And The “Art of the Possible”
For quite some time IT organizations have used automation and orchestration tooling to drive server and application deployments. By adding intelligence around when to trigger orchestrations, it’s possible to bring auto-scaling to your on-premise application deployments today. You can also make auto-scaling in the public cloud even more intelligent. In this blog post, we’ll discuss how we can use a tool like Cisco AppDynamics to feed ITSM and orchestration frameworks to to drive this auto-scaling behavior for on-premises workloads today.
The contents of this blog and the attached video demonstrate the types of technologies and approach to enterprise cloud that AHEAD is talking about and implementing every day. No one technology stack is going to let clients “deploy enterprise cloud.” Integration of key tools, processes, and approaches to adopting and adapting new technology are critical to success in the brave new cloud world. This overview and demonstration tie together all four of the fundamental pillars of the Cloud Management and Operations stack within the AHEAD Cloud Delivery Framework. These four pillars are Enterprise Service Management, Cloud Management Platform, Configuration Management, and Application Performance Management and they are critical to a well orchestrated and automated cloud platform. This write-up and demonstration will show one way in which this can be accomplished.
If you don’t feel like reading through the post and want to hear our expert talk through the process, here is a video “live from our lab”:
To illustrate this approach, we first need an application. AHEAD has developed a bag tracking application as a part of our fictitious company called AHEAD Aviation. This application (composed on microservices) uses a backend API that is queried to get results from our underlying bag database (running on MongoDB). This application is all running on-premises today in our AHEAD Aviation data center and managed as a business service in ServiceNow. In fact, enough explaining, here’s a nice diagram of the service mapping that’s generated by ServiceNow:
Zooming out a bit we can see how this relates to our on-premises VMware infrastructure as well:
Now that we have an idea of the components that make up the API service, we need an idea of what it needs to scale. In this case to scale and serve more API requests, additional VM application servers are needed. In the above figure server app-sr150 and app-sr160 represent that portion of the application.
Feedback Loop Overview – Understanding When To Trigger An Orchestration
Knowing when to trigger an orchestration event from application-centric feedback loop is really what drives this use case. For our AHEAD Aviation baggage application, we know that our app servers are designed to address about 1500 transactions per minute. By monitoring the number of transactions that are hitting our application concurrently, we can proactively scale our application layer so that our users aren’t impacted by this increased demand. While we’ve chosen to monitor number of transactions, more traditional aspects can be monitored as well, such as average transaction response time, cpu, or memory load on hosts.
Cisco AppDynamics As The Engine For The Feedback Loop
To monitor appropriately, we’ll leverage Cisco’s AppDynamics which is an application performance monitoring tool. AppDynamics is a SaaS based tool that enables transaction level application monitoring. It accomplishes this by leveraging agents that can look at code execution time of your app between each component of your application. By collating this data in their cloud platform and allowing you to set thresholds on key business transactions it enables you to trigger alerts or actions in response to breaches of these thresholds. It can also help application owners understand what normal and historical performance of the application look like by monitoring over time providing insight into application change as code is updated or as load increases. Here’s a look at what our Bag API performance looks like in the engine:
As you can see in this example, our two application servers are addressing just a bit more than 3000 requests per minute. This has thrown our business transaction health past a threshold in AppDynamics platform. In response, AppDynamics reaches out to our ServiceNow REST API to let ServiceNow manage and audit the process of updating our business service.
ServiceNow – Managing Our Business Service
As a part of rolling out application auto-scaling to the bag application, we’ve tested the process of deploying new application servers to our service using automation. Because we’ve tested, we’re confident enough in our orchestration to make these changes as “standard” or pre-approved changes in our ITSM tool avoiding manual approval and thereby increasing our ability to recover quickly from increased application load. That said, it’s important for us to still track and maintain an audit record of these changes to our application in our ITSM tooling. For this reason, we call our ITSM tooling to manage the process instead of directly reaching out to our orchestration engine.
Here’s the change request that we’ve automatically opened in ServiceNow:
Cisco UCS-Director – Providing The Automation Glue To Tie Our Process Together
After ServiceNow opens a standard change it reaches out and kicks off a VM build from UCS-Director. This workflow is a linear workflow that rolls through several tasks. First, it clones a new VM in our VMware environment, getting appropriate IP addresses, placing the VM on the appropriate storage and networking, and finally, calling our Puppet infrastructure to deploy our application code to the newly cloned machine. Here’s a quick view into that process in our Cisco UCS-D workflow designer:
Why Application Dependency Mapping Matters
After this process completes, our ServiceNow application dependency view is updated with our newly provisioned application servers. If you look closely, you’ll notice that we’ve purposely created a problem with our application to illustrate the value of what ServiceNow Dependency mapping provides us. While our APM tooling is looking at performance and orchestration framework provides the exact same builds to our environment over and over, it should be noted that all of our application machines are running on the same physical hypervisor host under the covers. Look closely at the image below and you’ll see that all of the app servers are sitting on iaasesx02 in our environment. Without service mapping, it would be difficult to see this relationship and an outage to a single host in our VMware infrastructure would cause a complete application outage! ServiceNow provides this insight regardless of underlying infrastructure. If this application were platformed on AWS EC2 instances, a relationship showing Availability Zone would be present.
Bundling It All Up
We’ve covered a lot in this post, before closing I’d like to first provide a quick recap on how we enabled this process:
- AppDynamics is aware that we want to trigger an action when the total number of transactions per minute exceeds our threshold
- AppDynamics realizes that the BagApp API is experiencing more transactions per minute than is normal for the current scale
- AppDynamics triggers an alert based on this workload change. This alert calls our ServiceNow REST API to open a standard (pre-approved) change for our business service
- ServiceNow triggers UCS-Director to fulfill the request
- UCS-Director deploys a new VM and passes this node off to Puppet to have the Bag API application code deployed
- Puppet deploys and enables the code on the newly created VM
- ServiceNow’s application dependency mapping is updated with our new application nodes
Want To Learn More?
This is just one example of many orchestrations that we’ve developed in the AHEAD Aviation lab. For more information on this process, cloud, or automation and orchestration, set up a briefing today.