When metadata is leveraged in Amazon Web Services (AWS), it allows organizations to have a better understanding of the state of their elastic public cloud environment. But what exactly is metadata, anyway? Think of it as one piece of data that describes another piece of data. For a real world example, pull your driver’s license out of your wallet. There are at least three dates visible: your date of birth, the license issue date, and the license expiration date. These numbers make sense when prefixed with the metadata classifications: DOB, Issued, and Expires. But can you imagine how difficult it would be to interpret these numbers without metadata classifications?
Similarly, an IT administrator would have a difficult time understanding how AWS resources are being consumed without key pieces of information like user, department, region, etc. AWS supports metadata classifications by allowing clients to tag resources with pieces of information like this. As a Cloud Services Engineer, I see most of our clients use tags to help gain a deeper understanding of a constantly elastic environment. The AHEAD public cloud team works with clients to provide best practice guidance and help determine the optimal tagging strategy to support their showback and chargeback requirements.
At this point in time, not all AWS services support tagging, including Amazon WorkSpaces. The WorkSpaces service allows organizations to quickly deliver cloud-based desktops to end users in a pay-as-you-go model. Recently, an AHEAD client utilizing the Consolidated Billing for AWS service needed a way to tie their WorkSpaces spend back to internal user IDs. Because metadata is so important and WorkSpaces doesn’t yet support tagging, our team created a custom application to capture the data needed to help them meet their chargeback requirements.
In creating this custom solution, the team had three primary objectives:
- Avoid third-party services. AWS is a broad platform with robust capabilities. We set out to create a solution using only services available within the AWS platform.
- Avoid manual work. We needed to build an automated solution that didn’t require ongoing effort and maintenance from the AWS administrator.
- Minimize ongoing costs. Because any query or function run in AWS consumes resources, we needed to create an efficient solution and avoid anything resource-intensive.
The core of the solution we developed is an application that is written in Python and runs via the Lambda serverless compute service. The application queries the WorkSpaces API, processes the returned data, and stores the appropriate WorkSpaces metadata in DynamoDB, another AWS tool and a fully-managed NoSQL database service. Lambda is scheduled to run the application on 15-minute intervals, which is frequent enough to capture an accurate state of any new or existing WorkSpaces. On the first day of each month, a detailed report of user WorkSpaces consumption is generated and stored in S3, an object-based, highly scalable and secure AWS storage solution.
We chose to utilize Lambda because it gives us access to compute resources and allows us to run code without the need to maintain additional infrastructure solely for the purpose of tracking resources. DynamoDB was chosen as the preferred storage backend due to extremely granular controls around cost, throughput, and storage.
The combination of the services chosen provided a self-contained, cost-effective, and scalable solution. It captured the metadata required by our client to help IT staff better understand how their WorkSpaces resources are being consumed, as well as provided the business with the required chargeback information.
If you’re interested in learning more about this custom WorkSpaces solution or the services we offer for AWS, feel free to email me directly at email@example.com.