Encryption and Decryption using AWS Key Management Service

Table of Contents

Introduction

Recently, I was working on getting a Terraform module set up to create resources to ingest AWS CloudWatch logs for Splunk. The process of preparing the HEC token used by Splunk required using the AWS Key Management Service (KMS) to encrypt the string. I learned quite a bit about how this process works and interacts with Terraform, so I thought I’d share some of the information that came out of it.

Prerequisites

These commands will only work with the AWS CLI v2. Most Linux package management systems still only have v1, so you may need to do a manual install. See this link for steps to install the latest version.

You will also need to have credentials set up to use AWS CLI. Typically this is done through setting up a profile in an ~/.aws/credentials file, using an AWS Access Key ID and Secret Access Key. See this page for a quick way to configure this using the aws configure command.

In my case, I’m testing using an AWS Cloud9 IDE instance, which sets this all up for you. I did have to upgrade the AWS CLI to v2 however.

AWS Key Management Service

Let’s start by talking about what AWS KMS does:

AWS Key Management Service (KMS) gives you centralized control over the cryptographic keys used to protect your data. The service is integrated with other AWS services making it easy to encrypt data you store in these services and control access to the keys that decrypt it. AWS KMS is also integrated with AWS CloudTrail, which provides you the ability to audit who used which keys, on which resources, and when.

This page outlines the basics, but to summarize for this use case, we are taking a plain text string (the Splunk HEC token), and using a KMS key to encrypt it and return a Base64 encoded payload.

Quick Terraform Sidebar

I won’t focus too much on Terraform in this post, but it is important to understand how the aws_kms_secrets data source works. The Terraform module expects a Base64 encoded, encrypted payload, as referenced in the documentation here:

payload - (Required) Base64 encoded payload, as returned from a KMS encrypt operation.

It is important to note, as above, that the AWS KMS encrypt operation by default will return a Base64 encoded string.

Creating the Required KMS Key

The AWS CLI can be used to create a KMS Key in AWS that can be used to encrypt the token. This can be done with the AWS CLI create-key command:

aws kms create-key \
--profile <AWS Profile Name> \
--tags TagKey=SplunkHECToken,TagValue=<Friendly Name> \
--description "KMS Key for storing Splunk HEC Token" \
--multi-region

The KMS Key data will be returned. Save the KeyID and ARN.

I would recommend adding an Alias to the key so it is easily visible in the AWS Console. This can be done with the AWS CLI KMS create-alias command:

aws kms create-alias \
--profile <AWS Profile Name> \
--alias-name alias/<Alias Name for AWS console> \
--target-key-id <KMS Key ID>

Encrypting the HEC Token – Plaintext

The HTTP Event Collector (HEC) Token is just an alphanumeric string with hyphens, so we can make one up for testing purposes:

abc123-def456-ghi789

To encrypt this string, use the following syntax:

aws kms encrypt \
--profile <AWS Profile Name> \
--key-id <KMS Key ID> \
--plaintext 'plain-text-hec-token' \
--cli-binary-format raw-in-base64-out \
--query CiphertextBlob \
--output text

The above command will return a CiphertextBlob payload string, which has been encrypted by the KMS key, and then Base64 encoded.

For details on the arguments used, see the aws kms cli encrypt documentation.

One interesting point is the –cli-binary-format raw-in-base64-out argument is needed, as otherwise the command will strip the hyphens from the string.

Encrypting the HEC Token – File

Another way to encrypt the HEC token is to specify it as a binary file. This gets around one of the problems that strips the hyphens from the string. This also may be helpful if you are retrieving the token from a secured source.

For this example, first, add the string to a file. This command will echo the string into a file, and ensure there is no newline character added:

echo -n <HEC Token> > ./hectoken.txt

Once you have the file, you can use this syntax to encrypt the token:

aws kms encrypt \
--profile <AWS Profile Name> \
--key-id <KMS Key ID> \
--plaintext fileb://hectoken.txt \
--query Ciphertextblob \
--output text

The fileb:// prefix instructs the CLI to read the data to encrypt, called the plaintext, from a file and pass the file’s contents to the command’s –plaintext parameter. More information is available in the AWS CLI encrypt command documentation.

Decrypting the HEC Token

To make sure that everything is working properly, you may want to test manually decrypting the payload with the KMS key, to ensure the output is what you expect to get. You can decrypt the payload using the AWS CLI decrypt command with the following syntax:

aws kms decrypt \
--profile <AWS Profile Name> \
--ciphertextblob 'Base64 encoded encrypted ciphertext' \
--query Plaintext \
--output text | base64 -d

One of the interesting things to note here, is that the output of the aws kms decrypt command is still Base64 encoded. This is why it needs to be passed to the base64 –decode function to arrive back at the original HEC token string. If you will be automating this, ensure that you have the base64 command available on the machine. In the above example, the payload is decrypted, and then piped to base64 to decode.

Demonstration Example

Now let’s see this all in action!

Creating the KMS Key:

The important thing in the above block is the KeyID value. You can also see that the Encryption Algorithm is SYMMETRIC_DEFAULT, which is the default so we won’t need to specify this later on, but it’s good to note, and can be if needed using the AWS CLI –encryption-algorithm argument.

Creating the KMS Alias:

Confirming in the AWS Console that the KMS key is active and has the alias set:

Encrypting the plaintext token into an encrypted, Base64 encoded string:

Finally, decrypting the string and Base64 decoding it:

Conclusion

As demonstrated, we were able to encrypt and decrypt the same string including the hyphens. Hopefully this sheds some light on ways to encrypt and decrypt data using the AWS Key Management Service (KMS).

Special thanks to AWS Support for clarifying a lot of this with me, and providing some of the options to tackle this use case.

Oh, and I’ve already disabled the key used in the example.