Graduate Program KB

AWS Workshop 03

Before you begin

We'll be using your AWS Sandbox accounts for this workshop.

  1. Make sure you have a an sso-session configuration section in your ~/.aws/config:
[sso-session j1]
sso_start_url = https://j1.awsapps.com/start
sso_region = ap-southeast-2
sso_registration_scopes = sso:account:access

If you don't, then you'll need to configure your sso-session section with the aws configure sso-session wizard:

aws configure sso
SSO session name: j1
SSO start URL [None]: https://j1.awsapps.com/start
SSO region [None]: ap-southeast-2
SSO registration scopes [None]: sso:account:access
  1. Next, we'll create and SSO linked developer profile for your sandbox account
aws configure sso
SSO session name (Recommended): j1

When prompted: - Choose your sandbox account from the list - Next, choose the SandboxDeveloper role - Set the CLI default client Region to ap-southeast-2 - CLI default output format to json

  1. Update the cli_pager setting in the pf-sandbox-developer profile:
aws configure set cli_pager "" --profile pf-sandbox-developer
  1. Verify that you have a [profile pf-sandbox-developer] section in your ~/.aws/config file:
grep -n -A6 "[profile pf-sandbox-developer]" ~/.aws/config
[profile pf-sandbox-developer]
sso_session = j1
sso_account_id = YOUR_SANDBOX_ACCOUNT_ID
sso_role_name = SandboxDeveloper
region = ap-southeast-2
output = json
cli_pager =
  1. Add an alias to your ~/.bash_aliases to log you in to your AWS sandbox account using your pf-sandbox-developer profile:
echo "alias pfsbd='aws sso login --profile pf-sandbox-developer'" >> ~/.bash_aliases
source ~/.bash_aliases
  1. If you've previously completed steps 1 and 2, refresh your pf-sandbox-developer token:
pfsbd
Attempting to automatically open the SSO authorization page in your default browser.
If the browser does not open or you wish to use a different device to authorize this request, open the following URL:

https://device.sso.ap-southeast-2.amazonaws.com/

Then enter the code:

ABCD-1234
Successfully logged into Start URL: https://j1.awsapps.com/start
  1. Verify that you can access resources in your AWS sandbox account using the pf-sandbox-developer profile:
aws ec2 describe-vpcs --profile pf-sandbox-developer --query Vpcs[0].VpcId
"vpc-xxxxxxxxxxxxxxx"

We'll be using the pf-sandbox-developer profile for the remainder of the workshop.

  1. Install jq
sudo apt-get install jq

Introduction to Cloudformation

In the previous workshop, we were introduced to the AWS CLI, which allowed us to script the creation and configuration of an S3 bucket to host a static website. We were also able to use the CLI to copy our website's resources into the bucket we created. In our homework, we extended our infrastructure to include a Cloudfront distribution, which we configured to securely serve requests to our S3 bucket. This approach required us to:

  1. Provision each resource indivdually
  2. Configure the resources to enable them to work together
  3. Clean up the resources once they were no longer required

In today's workshop we are going to see how we can provision and configure these resources using Cloudformation templates. A Cloudformation template describes all of our resources and their properties. When we deploy a Cloudformation template a Cloudformation stack is created. Deleting our resources is as simple as deleting the Cloudformation stack. This allows us to manage all of our resources as a single unit. Our scripts would need to be extended to allow users to specify the target region(s), but with Cloudformation, a template can be reused to repeatably deploy our resources to any region required. Finally, it's not easy to predict the resulting change to our deployed resources should we update our CLI scripts - with Cloudformation, we can see what the resulting change to our deployed resources will be before we commit to releasing it.

Next, let's talk about the core building blocks of Cloudformation in more detail: Templates, Stacks and Change sets.

Templates

Templates are JSON or YAML text files. They can have a .json, .yaml, .template or .txt suffix. You can think of a template as a recipe or blueprint for the AWS resources you would like to create. Here's a template that describes an S3 bucket:

{
  {
    "AWSTemplateFormatVersion": "2010-09-09",
    "Description": "Baby's First Template",
    "Resources": {
        "BabysFirstBucket":{
            "Type" : "AWS::S3::Bucket",
            "Properties" : {
            "BucketName" : "My little bucket",
            "Tags" : [ "delete-me","babys-first-bucket","boop-be-bo-bucket-beep","tag-this"],
             }
        }
      }
    }
}

The the top level, template specifies the following fields:

Lets look at the anatomy of the Resources section:

"Resources" : {
    "Logical ID" : {
        "Type" : "Resource type",
        "Properties" : {
            Set of properties
        }
    }
}

In our case the Logical ID of our S3 bucket is "BabysFirstBucket". The Type of our resource is "AWS::S3::Bucket". Under the Properties section we have specified:

  • BucketName - This is the name of the bucket
  • Tags - This is a list of tags to apply to the bucket

This Cloudformation template describes only a single resource, but the power of templates lies in their ability to describe multiple, interrelated resources. Templates can also accept input parameters, that can be specified when creating a stack; as an example, we could accept a parameter for the bucket name, thereby allowing the template to be reused in different contexts.

Stacks

A stack is a group of related resources. To create,update and delete a group of related resources, we create, update and delete stacks. A template defines all of the resources that form a stack. To create the resources described by a template, we submit our template to the Cloudformation service. The Cloudformation service will then create a stack and provision all of the resources described in the submitted template as part of the newly created stack.

Change sets

A change set is a summary of the changes that would be made to resources provisioned by a stack. Change sets allow us to see what the net effect of our changes will be before we make them.

Deploying a Statically Hosted Website with S3 using Cloudformation

Creating a Cloudformation Template

Let's begin by creating new, empty template:

touch my-statically-hosted-website.template.json

Next, let's add a new Resource block to our template and define an S3 bucket resource:

{
  "Resources": {
    "MyStaticallyHostedWebsiteBucket": {
      "Type": "AWS::S3::Bucket"
    }
  }
}

The name we've used for our resource (MyStaticallyHostedWebsiteBucket) is a logical name. When Cloudformation creates the resource, it will generate a name physical name based on a combination of the:

  • Logical name
  • Stack name
  • A unique identifier

Generating, viewing and deploying Cloudformation Change sets

To deploy the resources described in our template, we need to use the AWS CLI.

Howewer, before we deploy our stack, let's create and review a change set. We can do this by invoking aws cloudformation deploy passing the --no-execute-changeset parameter.

aws cloudformation deploy --stack-name my-statically-hosted-website-stack --template-file ./my-statically-hosted-website.template.json --profile pf-sandbox-developer --no-execute-changeset;

Waiting for changeset to be created..
Changeset created successfully. Run the following command to review changes:
aws cloudformation describe-change-set --change-set-name arn:aws:cloudformation:ap-southeast-2:0000000000:changeSet/awscli-cloudformation-package-deploy-0000000/0000-0000-0000-0000-000000000

Take note of the last line of the output above. We can then use the above command to view a description of the generated change set:

aws cloudformation describe-change-set --change-set-name arn:aws:cloudformation:ap-southeast-2:0000000000:changeSet/awscli-cloudformation-package-deploy-0000000/0000-0000-0000-0000-000000000 --profile pf-sandbox-developer
{
    "Changes": [
        {
            "Type": "Resource",
            "ResourceChange": {
                "Action": "Add",
                "LogicalResourceId": "MyStaticallyHostedWebsiteBucket",
                "ResourceType": "AWS::S3::Bucket",
                "Scope": [],
                "Details": []
            }
        }
    ],
    "ChangeSetName": "awscli-cloudformation-package-deploy-0000000",
    "ChangeSetId": "arn:aws:cloudformation:ap-southeast-2:541172580536:changeSet/awscli-cloudformation-package-deploy-0000000/0000-0000-0000-0000-000000000",
    "StackId": "arn:aws:cloudformation:ap-southeast-2:541172580536:stack/my-statically-hosted-website-stack/7c9afec0-cde9-11ed-95c9-06721abef9c6",
    "StackName": "my-statically-hosted-website-stack",
    "Description": "Created by AWS CLI at 2023-03-29T04:23:40.654068 UTC",
    "Parameters": null,
    "CreationTime": "2023-03-29T04:23:42.133000+00:00",
    "ExecutionStatus": "AVAILABLE",
    "Status": "CREATE_COMPLETE",
    "StatusReason": null,
    "NotificationARNs": [],
    "RollbackConfiguration": {},
    "Capabilities": [],
    "Tags": null,
    "ParentChangeSetId": null,
    "IncludeNestedStacks": false,
    "RootChangeSetId": null
}

Looking at the output we can see a section called Changes.

    "Changes": [
        {
            "Type": "Resource",
            "ResourceChange": {
                "Action": "Add",
                "LogicalResourceId": "MyStaticallyHostedWebsiteBucket",
                "ResourceType": "AWS::S3::Bucket",
                "Scope": [],
                "Details": []
            }
        }
    ],

This is a list of all the changes that will result from applying the Change set. Each Change has a Type (currently there is only one type supported: Resource) which describes the type of entity that this change affects. The ResourceChange property describes the resource and the action that will be performed if this Change set is applied.

In this case we will be adding a new S3 bucket resource with a logical resource identifier of "MyStaticallyHostedWebsiteBucket".

To deploy our changes by executing the change set we created above:

aws cloudformation execute-change-set --change-set-name arn:aws:cloudformation:ap-southeast-2:0000000000:changeSet/awscli-cloudformation-package-deploy-0000000/0000-0000-0000-0000-000000000 --profile pf-sandbox-developer

We should now have a new stack called my-statically-hosted-website-stack:

aws cloudformation describe-stacks --stack-name my-statically-hosted-website-stack --profile pf-sandbox-developer
{
    "Stacks": [
        {
            "StackId": "arn:aws:cloudformation:ap-southeast-2:00000000000:stack/my-statically-hosted-website-stack/0000-0000-0000-0000-000000000",
            "StackName": "my-statically-hosted-website-stack",
            "ChangeSetId": "arn:aws:cloudformation:ap-southeast-2:00000000000:changeSet/awscli-cloudformation-package-deploy-0000000000000/0000-0000-0000-0000-000000000",
            "CreationTime": "2023-03-29T04:23:42.133000+00:00",
            "LastUpdatedTime": "2023-03-29T05:10:23.389000+00:00",
            "RollbackConfiguration": {},
            "StackStatus": "CREATE_COMPLETE",
            "DisableRollback": false,
            "NotificationARNs": [],
            "Tags": [],
            "EnableTerminationProtection": false,
            "DriftInformation": {
                "StackDriftStatus": "NOT_CHECKED"
            }
        }
    ]
}

To see what resources were created by our stack:

aws cloudformation describe-stack-resources --stack-name my-statically-hosted-website-stack --profile pf-sandbox-developer
{
    "StackResources": [
        {
            "StackName": "my-statically-hosted-website-stack",
            "StackId": "arn:aws:cloudformation:ap-southeast-2:00000000000:stack/my-statically-hosted-website-stack/0000-0000-0000-0000-000000000",
            "LogicalResourceId": "MyStaticallyHostedWebsiteBucket",
            "PhysicalResourceId": "my-statically-hosted-web-mystaticallyhostedwebsit-000000000000",
            "ResourceType": "AWS::S3::Bucket",
            "Timestamp": "2023-03-29T05:10:48.342000+00:00",
            "ResourceStatus": "CREATE_COMPLETE",
            "DriftInformation": {
                "StackResourceDriftStatus": "NOT_CHECKED"
            }
        }
    ]
}

Cloudformation Outputs

The Outputs object in a template allows us to declare values that we would like to make available after the stack is created. For example, if we want our stack to output the domain name of the S3 bucket it creates, we need to:

  1. Get the value of the domain name attribute of the S3 bucket
  2. Publish the value of domain name attribute as an output of the deployed stack

We can use the Fn::GetAtt intrinsic function to get the value of the DomainName attribute of our S3 bucket.

The Fn:GetAtt intrinsic function requires two parameters:

  1. The logical name of the resource to query
  2. The name of the attribute we want the value of
{ "Fn::GetAtt": ["logicalNameOfResource", "attributeName"] }

The Outputs section of a Cloudformation template allows you to declare up to 200 outputs per template:

"Outputs" : {
  "Logical ID" : {
    "Description" : "Information about the value",
    "Value" : "Value to return",
    "Export" : {
      "Name" : "Name of resource to export (NOTE: 'Export' only required for cross-stack references)"
    }
  }
}

To export the DomainName attribute of the S3 bucket with the logical id MyStaticallyHostedWebsiteBucket, we need to add the following section to our template:

{
  "Outputs": {
    "S3BucketDomainName": {
      "Value": {
        "Fn::GetAtt": ["MyStaticallyHostedWebsiteBucket", "DomainName"]
      }
    }
  }
}

Let's update our template so it has the following following content:

{
  "Resources": {
    "MyStaticallyHostedWebsiteBucket": {
      "Type": "AWS::S3::Bucket"
    }
  },
  "Outputs": {
    "S3BucketDomainName": {
      "Value": {
        "Fn::GetAtt": ["MyStaticallyHostedWebsiteBucket", "DomainName"]
      }
    }
  }
}

aws cloudformation update-stack --stack-name my-statically-hosted-website-stack --profile pf-sandbox-developer --template-body '{"Resources": {"MyStaticallyHostedWebsiteBucket": {"Type":"AWS::S3::Bucket"}},"Outputs": {"S3BucketDomainName": {"Value": {"Fn::GetAtt" : [ "MyStaticallyHostedWebsiteBucket", "DomainName" ]}}}}'
{
    "StackId": "arn:aws:cloudformation:ap-southeast-2:00000000000:stack/my-statically-hosted-website-stack/0000-0000-0000-0000-000000000"
}

If we describe the stack again, we should now see the S3BucketDomainName key and it's corresponding value my-statically-hosted-web-mystaticallyhostedwebsit-00000000000.s3.amazonaws.com under Outputs:

aws cloudformation describe-stacks --stack-name my-statically-hosted-website-stack --profile pf-sandbox-developer
{
    "Stacks": [
        {
            "StackId": "arn:aws:cloudformation:ap-southeast-2:00000000000:stack/my-statically-hosted-website-stack/0000-0000-0000-0000-000000000",
            "StackName": "my-statically-hosted-website-stack",
            "CreationTime": "2023-03-29T04:23:42.133000+00:00",
            "LastUpdatedTime": "2023-03-29T08:30:01.212000+00:00",
            "RollbackConfiguration": {},
            "StackStatus": "UPDATE_COMPLETE",
            "DisableRollback": false,
            "NotificationARNs": [],
            "Outputs": [
                {
                    "OutputKey": "S3BucketDomainName",
                    "OutputValue": "my-statically-hosted-web-mystaticallyhostedwebsit-00000000000.s3.amazonaws.com"
                }
            ],
            "Tags": [],
            "EnableTerminationProtection": false,
            "DriftInformation": {
                "StackDriftStatus": "NOT_CHECKED"
            }
        }
    ]
}

Handling user input

We can declare parameters in a template's Parameters object. A parameter contains a list of attributes that define it's value and any constraints that apply to its value.

Let's add a CustomTag parameter to our template file: my-statically-hosted-website.template.json. This parameter allow users to apply a custom tag to the S3 bucket:

{
  "Parameters": {
    "CustomTag": {
      "Description": "A tag of your own choosing",
      "Type": "String",
      "MinLength": 1,
      "MaxLength": 256,
      "Default": "Do your own tagging please..."
    }
  },
  "Resources": {
    "MyStaticallyHostedWebsiteBucket": {
      "Type": "AWS::S3::Bucket",
      "Properties": {
        "Tags": [{ "Key": "CustomTag", "Value": { "Ref": "CustomTag" } }]
      }
    }
  },
  "Outputs": {
    "S3BucketDomainName": {
      "Value": {
        "Fn::GetAtt": ["MyStaticallyHostedWebsiteBucket", "DomainName"]
      }
    }
  }
}

Let's deploy the changes we've made to our existing stack:

aws cloudformation deploy --stack-name my-statically-hosted-website-stack --template-file ./my-statically-hosted-website.template.json --parameter-overrides CustomTag=BestTagEvar --profile pf-sandbox-developer;

Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - my-statically-hosted-website-stack

We should now verify that our newly created bucket has been tagged correctly:

aws s3api list-buckets  --profile pf-sandbox-developer | jq '.Buckets[0].Name' | xargs -I{} aws s3api get-bucket-tagging --bucket {} --profile pf-sandbox-developer;
{
    "TagSet": [
        {
            "Key": "aws:cloudformation:stack-name",
            "Value": "my-statically-hosted-website-stack"
        },
        {
            "Key": "CustomTag",
            "Value": "BestTagEvar"
        },
        {
            "Key": "aws:cloudformation:logical-id",
            "Value": "MyStaticallyHostedWebsiteBucket"
        },
        {
            "Key": "aws:cloudformation:stack-id",
            "Value": "arn:aws:cloudformation:ap-southeast-2:00000000000:stack/my-statically-hosted-website-stack/0000-0000-0000-0000-000000000"
        }
    ]
}

Cleaning up

We can use the delete-stack command to delete the my-statically-hosted-website-stack and all it's related resources:

aws cloudformation delete-stack --stack-name my-statically-hosted-website-stack --profile pf-sandbox-developer;

We can verify that it has been delete by using describe-stacks:

aws cloudformation describe-stacks --stack-name my-statically-hosted-website-stack --profile pf-sandbox-developer;

An error occurred (ValidationError) when calling the DescribeStacks operation: Stack with id my-statically-hosted-website-stack does not exist

Homework

In AWS Workshop 02 we scripted the deployment of a static website using a Cloudfront distribution and an S3 bucket.

For this workshop, you need to write a Cloudformation template to deploy a Cloudfront distribution that serves the content for a static, single page application hosted in an S3 bucket.

Stretch goal: Can you use Cloudformation to deploy the contents of your single page application to an S3 bucket? (Hint: could you use a Lambda? What about custom resources?)