Graduate Program KB

  1. What is Cloud Computing?
  2. IAM (Identity and Access Management)
  3. EC2 (Elastic Compute Cloud)
    1. Security Groups
    2. Setting Up and Connecting to EC2 Instances
    3. EC2 Instance Purchasing Options
    4. EC2 Shared Responsibility
  4. EC2 Instance Storage
    1. EBS
    2. Automation of Instance Creation
    3. EFS
    4. FSx
    5. EC2 Storage Shared Responsibility
  5. Elastic Load Balancing & Auto Scaling Groups
    1. Load Balancing
  6. Amazon S3
    1. S3 Storage Classes
    2. S3 Shared Responsibility
    3. AWS Snow Family
  7. Databases & Analytics
  8. Other Compute Services
    1. ECS (Elastic Container Service), Fargate & ECR
    2. Serverless
    3. Batch, Lightsail
  9. Deployments & Managing Infrastructure at Scale
  10. Leveraging the AWS Global Infrastructure
  11. Cloud Integrations
  12. Cloud Monitoring
  13. VPC and Networking
  14. Security and Compliance
  15. Machine Learning
  16. Account Management, Billing & Support
    1. Account Management
    2. Pricing
    3. Billing and Costing
    4. Support
  17. Advanced Identity
  18. Other AWS Services
    1. Migration
  19. AWS Architecting & Ecosystem
    1. Six Pillars of Well-Architected Frameworks
    2. AWS Cloud Adoption Framework
    3. Ecosystem

What is Cloud Computing?

  • Servers are composed of:
    • Compute (CPU)
    • Memory (RAM)
    • Storage (Data)
    • Database (Structured storage of data)
    • Network (Routers, switch, DNS server)
  • Router: Networking device that forwards packets between computer networks
  • Switch: Takes packets and sends them to the correct device on your network
  • Cloud Computing: On-demand delivery of computer power, database storage, applications, other IT resources.
  • Cloud deployment models
    • Private Cloud: Used by a single organization, who has complete control over the service. Provides security for sensitive application and can be used to meet that org's specific business needs. eg. Rackspace
    • Public Cloud: Cloud resources owned by a third-party service delivered over the internet. eg. Azure, GCP and AWS
    • Hybrid Cloud: Maintain some bare-metal services and extend their capabilities with cloud services. Allows control over sensitive assets in private infrastructure.
  • Five Characteristics of Cloud Computing:
    • On-Demand self service
    • Broad network access - Resources available over the network through diverse client platforms
    • Multi-tenancy and resource pooling
    • Rapid elasticity and scalability - Automatically and quickly scale up and down based on demand
    • Measured service - Usage is measured and customers pay for what they use
  • Six Advantages of Cloud Computing:
    • Trade capital expense (CAPEX) for operational expense (OPEX). Reduced Total Cost of Ownership (TCO)
    • Benefit from massive economies of scale
    • Stop guessing capacity (Scale based on actual usage)
    • Increase speed and agility
    • Stop spending money running and maintaining data centers
    • Leverage the AWS global infrastructure to go global quickly
  • Problems solved by cloud
    • Flexibility
    • Cost-Effectiveness
    • Scalability
    • Elasticity
    • High-availability and fault-tolerance
    • Agility
  • Types of Cloud Computing
    • Infrastructure as a Service (IaaS) - Provides building blocks for cloud IT (raw network, compute and data storage space)
      • eg. AWS EC2
      • eg. GCP, Azure, Rackspace, Digital Ocean, Linode
    • Platform as a Service (PaaS) - Removes the need for org to manage underlying infrastructure, allowing them to focus on deployment and management of apps
      • eg. AWS Elastic Beanstalk
      • eg. Heroku, GCP Google App Engine, Windows Azure
    • Software as a Service (SaaS) - Completed IT product that is run and managed by the cloud service provider
      • eg. AWS Rekognition for Machine Learning (among many other services)
      • eg. Gmail, Dropbox, Zoom
  • Pricing of the Cloud
    • Compute time
    • Data stored in cloud
    • Data transfer out of cloud (data transfer in is free)
  • AWS Background
    • AWS Cloud History
      • 2002 - Internal launch
      • 2004 - Public launch of SQS
      • 2006 - Public launch of SQS, S3 and EC2
      • 2007 - European launch
    • AWS accounts for 47% of the cloud market in 2019, followed by Microsoft at 22% market share
    • Over 1,000,000 active users
    • Use cases: Enterprise IT, Backup & Storage, Big Data Analytics, Web hosting, Mobile Apps, Gaming Server Hosting
  • AWS Global Infrastructure is comprised of:
    • Regions
      • eg. ap-southeast-2 for Asia Pacific (Sydney).
      • Clusters of data centers. Most AWS services are scoped to a region.
      • Choose region based on:
        • Compliance with policy and requirements
        • Proximity to customers
        • The services available in each region
        • Pricing
    • Availability Zones
      • eg. ap-southeast-2a, ap-southeast-2b, ap-southeast-2c
      • Each AZ consists of one or more discrete data centers with redundant power, networking, connectivity. Completely separate from each other.
    • Data Centers
    • Edge Locations / Points of Presence
      • AWS has 400+ Points of Presence (400+ Edge Locations & 10+ Regional Caches) in 90+ cities across 40+ countries
      • Used to deliver content to end users with low latency
  • AWS Console Home shows:
    • Recently visited services
    • Health issues
    • Cost and usage info for your account
    • Tutorials for starting up solutions in several different services
  • Services Drop-down shows all services listed alphabetically or by category
  • Search bar allows users to search for services, features, documentation, knowledge articles, tutorials, etc.
  • See AWS Services by Region here
  • Shared Responsibility Model - Model for shared security responsibility between AWS and its customers
    • AWS is responsible for the security of its software and global infrastructure
    • Customers are responsible for the security of:
      • Customer data
      • Platform, app, identity and access management
      • OS, network and firewall configuration
      • Client-side data encryption & data integrity authentication
      • Server-side encryption
      • Network traffic protection (encryption, integrity, identity)

IAM (Identity and Access Management)

  • IAM: Allows creation and grouping of users by role and responsibility. Offered as a global service.
  • Groups can only contain users, not other groups.
  • Users can belong to 0+ groups at a time.
  • Apply the principle of least privilege. Users and groups should only have the permissions they need to perform their role.
    • For this reason avoid ever using the root account
  • Sign-in URL can be customized for IAM users
  • Users/Groups can be assigned permissions via policies in the form of JSON documents, eg.
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "ec2:Describe",
      "Resource": "*"
    }
  ],
}
  • IAM Policies Structure:
    • "Version" - Policy language version. Always set to "2012-10-17"
    • "Id" - Policy identifier (optional)
    • "Statement" - One or more individual statements (array). Consists of:
      • "Sid" - Statement identifier (optional)
      • "Effect" - Allow or deny
      • "Principal" - Account/user/role this policy applies to
      • "Action" - List of actions this policy concerns
      • "Resource" - List of resources this policy concerns
      • "Condition" - Conditions for when the policy is in effect
  • Permissions within policies can be edited via a form of via editing the raw JSON
  • Wildcards (*) can be used to apply permissions to multiple actions, resources, etc.
  • IAM allows setup of account/login policies including:
    • Password policies (minimum length, character sets, password expiry, whether they can change their own password, prevent re-use, etc)
    • MFA policies (Auth app / Yubikey / Hardware Fob)
  • Options for users to access AWS
    • AWS Management Console (password + MFA protected)
    • AWS CLI (access keys)
    • AWS SDK (access keys)
  • Access keys are generated through the AWS console and are managed by users
    • Access Key ID is like your username
    • Secret Access Key is like your password
  • AWS CLI command format: aws [options] <command/service> <subcommand> [parameters]
  • Alternative to AWS CLI is AWS CloudShell, allowing you to issue bash and AWS CLI commands in the browser.
    • Access by clicking the terminal icon in the top bar
    • Provides a linux environment with persistent files
  • Services can be provided permissions via assigned IAM Roles
  • Security Tools
    • IAM Credentials Report (account-level reporting)
    • IAM Access Advisor (user-level reporting). Can use this information to revise your policies
  • IAM Guidelines
    • Don't use the root account except for AWS account setup
    • One physical user = One AWS user
    • Assign users to groups and assign permissions to groups
    • Create a strong password policy
    • Use and enforce the use of MFA
    • Create and use Roles for giving permissions to AWS services
    • Use Access Keys for Programmatic Access (CLI/SDK)
    • Audit permissions of your account using IAM Credentials Report & IAM Access Advisor
    • Never share IAM users & Access Keys
  • Shared Responsibility Model for IAM
    • AWS is responsible for:
      • Infrastructure (global network security)
      • Configuration and vulnerability analysis
      • Compliance validation
    • You are responsible for:
      • Users, groups, roles, policies management and monitoring
      • Enabling MFA
      • Rotation of keys
      • Using IAM tools to apply appropriate permissions
      • Analyzing access patterns & reviewing permissions

EC2 (Elastic Compute Cloud)

  • EC2 - AWS IaaS. Capabilities:
    • Renting virtual machines (EC2)
    • Storing data on virtual drives (EBS)
    • Distributing load across machines (ELB)
    • Scaling the services using an auto-scaling group (ASG)
  • EC2 Sizing & configuration options
    • OS (Windows, Linux or MacOS)
    • How much compute power & how many cores
    • How much RAM
    • How much storage space
    • Type of netwrok attached (speed, IP address)
    • Firewall rules (Security Groups)
    • Bootstrap script to configure at first launch (User Data).
  • Commands in the EC2 user data will be run as root and can automate tasks like:
    • Software installation/updates
    • Downloading files from the internet
    • Any other administrative tasks
  • Instance types:
    • EC2 instance types have a consistent naming convention. For the m5.2xlarge instance type:
      • m is the instance class
      • 5 is the generation of the instance class
      • 2xlarge represents the size of the instance class
    • Instance types may either be general purpose (eg. t2.micro) or optimized for compute, memory or storage optimized. Think about what you need from the instance.
    • Compare instance types with Vantage

Security Groups

  • Security Groups act as a firewall controlling both inbound and outbound traffic for EC2 instances, and are fundamental to network security in AWS.
    • Can be attached to multiple instances
    • Are locked down to a region/VPC combination
    • Lives 'outside' the EC2 (if traffic is blocked, the instance won't know about it)
    • Good practice to maintain a separate security group for SSH access
    • If your application is not accessible, it's a security group issue
    • If your application refuses connection, it's an application issue
    • All inbound traffic is blocked by default, while all outbound traffic is authorised by default
  • Refresher on well-known ports:
    • 22 - SSH
    • 21 - FTP
    • 22 - SFTP
    • 80/8080/8008 - HTTP
    • 443 - HTTPS
    • 4489 - RDP (Remote Desktop Protocol) for Windows instances

Setting Up and Connecting to EC2 Instances

  • Go to Instances > Launch an Instance and tailor the options to fit your needs.
    • Be sure to create a key pair in .pem format to allow access to your instance.
  • Connect to your instance with the command ssh -i <private-key>.pem ec2-user@<instance-ip>. This requires the instance's security group to accept traffic from your IP over port 22.
  • If you are unable to access your instance despite the fact your security group is written to permit the type of traffic you're using, try the following steps:
    • View details about the instance. If there is no public IP set for this instance, note down the instance's subnet ID.
    • Go to the VPC service dashboard.
    • Go to Virtual private cloud > Internet Gateways and create an Internet Gateway. Associate it with the VPC of the same ID you noted down before. Note down the Internet Gateway ID.
    • Go to Virtual private cloud > Route tables and, for the table associated with the subnet ID you noted down before, edit the route table to add a route where the 'Destination' is 0.0.0.0/0 and the target is the Internet Gateway of the same ID you entered in before. Ensure that the subnet route table is explicitly associated with the subnet.
    • Internet traffic to resources on the subnet should now be routed properly.
  • You can also connect to the instance through EC2 Instance Connect
    • First add the IP range Amazon uses for EC2 Instance Connection in your region to your security group rules for SSH traffic. You can find Amazon's IP ranges here. At time of writing, the EC2 IC range in the ap-southeast-2 region is 13.239.158.0/29.
    • Go to the list of instances, click on your instance and then click the "Connect" button.
  • To allow the instance to perform AWS queries via the CLI, assign it an IAM Role that inherits the permissions it needs

EC2 Instance Purchasing Options

  • On-Demand Instances - Short-term, uninterrupted workload, predictable pricing, pay by the second
  • Reserved (1 or 3 Years) - Reserve specific instance attributes, including instance type, region, tenancy and OS.
    • Reserved Instances for long workloads
    • Convertible Reserved Instances for long workloads with flexible instances
    • The longer reservation period of 3 years
  • Savings Plan (1 or 3 Years) - Commit to an amount of usage instead of to a specific instance type. For long workloads.
    • Gets you a discount based on long term usage (up to 72%)
    • Usage beyond EC2 Savings Plan billed at On-Demand price
    • Locked to a specific instance family & AWS Region
  • Spot Instances for short workloads. Cheap, but not reliable.
    • Can get a discount of up to 90% compared to On-demand
    • The most cost-efficient instances in AWS
    • No suitable for critical jobs or databases
  • Dedicated Hosts - Book an entire physical server and control instance placement
    • Allows you to address compliance requirements and use your existing server-bound software licenses (per-socket, per-core, per-VM software licenses)
    • On-demand or Reserved (1 or 3 years)
  • Dedicated Instances - Instances run on hardware dedicated to you. May share hardware with other instances in the same account
  • Dedicated Hosts - Same as dedicated instances, but gives you access to the physical server and makes hardware visible from the instance.
  • Capacity Reservations - Reserve capacity in a specific AZ for any duration
    • No time commitment, no billing discounts
    • Combine with Regional Reserved Instances and Savings Plans to benefit from billing discounts
    • Charged at on-demand rate whether you run instances or not
    • Suitable for short-term uninterrupted workloads in a specific AZ

EC2 Shared Responsibility

  • Amazon:
    • Infrastructure (global network security)
    • Isolation on physical hosts
    • Replacing faulty hardware
    • Compliance validation
  • You:
    • Security Groups rules
    • OS patches and updates
    • Software and utilities installed on the EC2 instance.
    • IAM Roles assigned to EC2 & IAM user access management
    • Data security on your instance

EC2 Instance Storage

EBS

  • Elastic Block Store (EBS) Volume - A networked storage drive you can attach to instances to allow them to persist data.
    • Can only be mounted to one instance at a time (unless you use a single Provisioned IOPS SSD (io1 or io2) volume. See more info here)
    • Bound to a specific AZ.
    • Free tier: 30GB of EBS storage per month. Either General Purpose (SSD) or Magnetic
    • As they use the network to communicate with instances, there may be some latency.
    • Have a provisioned capacity (in GBs and IOPS). You get billed for this provisioned capacity. It can be increased over time.
  • By default, the root EBS volume for an EC2 instance is deleted when the instance terminates, but any other attached EBS volume is not. This behaviour can be controlled through the AWS console
  • EBS Snapshots - Used to make a backup of your EBS volume at a point in time.
    • Allows copying of volume data across AZs. New volumes can be created from snapshots copied to the new AZ.
  • EBS Snapshot Archive - Move a snapshot to an archive tier (75% cheaper). Takes within 24 to 72 hours for restoring the archive.
  • Recylce Bin - Setup rules to retain EBS snapshots and AMIs after deletion to allow for recovery after accidental deletion. Retention period can be specified to be any time from 1 day to 1 year.

Automation of Instance Creation

  • Amazon Machine Image (AMI) - Customization of an EC2 instance, with your own added software, config, OS, etc. Results in a faster boot/configuration time as all your software is pre-packaged

    • Bound for a specific region, but can be copied across regions
  • You can launch EC2 instances from a public AMI (AWS provided), your own AMI, or an AWS Marketplace AMI

  • AMI Process:

    1. Start an EC2 instance and customize it
    2. Stop the instance to preserve data as-is
    3. Build an AMI (will also create an EBS snapshot)
    4. Launch instances from the AMI by selecting it in the Application and OS Images section when creating a new instance.
  • EC2 Image Builder - Automate the creation, maintenance and testing of EC2 AMIs

    • Can be run on a schedule, or whenever packages are updated
    • Free service (only pay for underlying resources)
  • EC2 Instance Store - High-performance hardware-connected EC2 storage alternative to network-connected EBSes.

    • Store is emphemeral, and will be lost upon termination of the associated instance.
    • Good for buffer/cache/scratch data/temporary content
    • Risk of data loss if hardware fails
    • Backups and replication are your responsibility.

EFS

  • Elastic File System (EFS) - Managed NFS (network file system) that can be mounted to many instances at once across multiple AZs.
    • Highly available, scalable, expensive, pay per use
    • Attached to instances via an EFS Mount Target
  • EFS Infrequent Access (EFS-IA)
    • Storage class cost-optimized for rarely accessed files
    • Up to 92% lower cost compared to standard EFS
    • EFS will automatically move files to EFS-IA based on last time they were accessed according to your defined Lifecycle Policy.
    • Transparent to applications accessing EFS.

FSx

Amazon FSx - Launch alternative third-party high-performance file systems on AWS. Currently includes:

  • FSx for NetApp ONTAP
  • FSx for OpenZFS
  • FSx for Windows File Server
  • FSx for Lustre
  • FSx for Windows File Server - Fully managed, reliable and scalable Windows native shared file system.
    • Supports SMB protocol & Windows NTFS
    • Integrate with Active Directory
    • Can be accessed from AWS or your on-premise infrastructure
  • FSx for Lustre - Fully managed, reliable and scalable file storage for High-Performance Computing
    • Linux based (Lustre = Linux + Cluster, apparently)
    • Use case = Machine Learning, Analytics, Video Processing, Financial Modeling etc
    • Scales up to 100GB/s, millions of IOPS

EC2 Storage Shared Responsibility

  • Amazon:
    • Infrastructure
    • Replication of data for EBS volumes & EFS drives
    • Replacing faulty hardware
    • Ensuring employees cannot access your data
  • You:
    • Setting up backup/snapshot procedures
    • Data encryption
    • Data on the drives
    • Understanding the risk of using EC2 Instance Store

Elastic Load Balancing & Auto Scaling Groups

  • Scalability - An application can accomodate for greater loads by increasing hardware power (scaling up; vertical scalability) or by adding nodes (scaling out; horizontal scalability)
  • Elasticity - Auto-scaling, so the system can scale based on the load
  • High Availability - Instances are available in multiple AZs

Load Balancing

  • Load Balancer - Servers that forward internet traffic to multiple EC2 instances downstream.
  • Why use a load balancer?
    • Spread load across multiple instances while exposing a single point of access to your application
    • Seamlessly handle failures of downstream instances
    • Do regular health checks to your instances
    • Provide SSL termination (HTTPS) for your websites
    • High availability across zones
  • Elastic Load Balancer (ELB) - Managed load balancer
    • 4 kinds of load balancers offered by AWS
    • Application Load Balancer (HTTP/HTTPS/gRPC)
      • HTTP Routing features
      • Use case: Static DNS
    • Network Load Balancer (TCP) - Layer 4
      • High Performance
      • Use case: Static IP through Elastic IP
    • Gateway Load Balancer (GENEVE on IP packets) - Layer 3
      • Use case: Route traffic to firewalls that you manage on EC2 instances
      • Use case: Intrusion detection via packet analysis
    • Classic Load Balancer - Layer 4 & 7 (retired in 2023)
  • Auto Scaling Group -
    • Allows you to automatically:
      • Scale out to match an increased load
      • Scale in to match a decreased load
      • Ensure a minimum and maximum number of machines
      • Register new instances to a load balancer
      • Replace unhealthy instances
    • Saves cost as you're always making sure you're running at optimal capacity.
    • Scaling Strategies:
      • Manual Scaling: Update the size of an ASG manualy
      • Dynamic Scaling: Respond to changing demand
        • Simple / Step Scaling - Scale when a CloudWatch alarm is triggered
        • Target Tracking Scaling - eg. target a specific average ASG CPU load of around 40%
        • Scheduled Scaling - Anticipate a scaling based on known usage patterns
        • Predictive Scaling - Use machine learning to predict future traffic ahead of time. Useful when your load has predictable time-based patterns

Amazon S3

  • S3 - Infinitely scaling storage. Used by many AWS services
    • Use cases:
      • Backup and storage
      • Disaster recovery
      • Archive
      • Hybrid Cloud storage
      • Application hosting
      • Data lakes & big data analytics
      • Software delivery
      • Static website
  • S3 Buckets - Essentially directories to store "objects" (files)
    • Buckets must have a globally unique name across all regions across all accounts
    • Buckets are defined at the region level
    • NAMING CONVENTION:
      • No uppercase or underscores
      • 3-63 chars long
      • Not an IP
      • Must start with a lowercase letter or number
      • Must NOT start with the prefixes xn-- or -s3alias
  • S3 Objects have keys, which are their full path
    • Composed of prefix + object name, eg s3://my-bucket/my_folder/another_folder/my_file
    • There are no directories within buckets, paths are just to make unique keys
    • Max size is 5TB
    • If uploading more than 5GB, must use "multi-part upload"
    • Has metadata (list of text key/value pairs for system/user metadata)
    • Can have tags (unicode key/value pairs up to 10), useful for security/lifecycle
    • Version ID, if versioning is enabled
  • S3 Security
    • User-based
      • IAM policies (which API calls allowed for a specific user from IAM)
    • Resource-Based
      • Bucket Policies - Bucket-wide rules from the S3 console. Allows cross account
      • Object ACL - Finer grain control, can be disabled
      • Bucket ACL - Less common, can be disabled
    • An IAM principal can access an S3 object if the user IAM permissions allow it OR the resource policy allows it AND there's no explicit deny rule
    • If EC2 instances need to be able to access objects in buckets, create an EC2 Instance Role and apply the relevant IAM permissions
    • Settings available for blocking public access (can be set at the account level)
    • If you want to allow anyone to access your bucket, edit your bucket policies and go to the policy generator, set the principal to *, set the GetObject action and set the ARN to arn:aws:s3:::<your-bucket-name>/*
  • S3 Static Website Hosting
    • URL will be (depending on the region):
      • http://<your-bucket-name>.s3-website-<aws-region>.amazonaws.com or
      • http://<your-bucket-name>.s3-website.<aws-region>.amazonaws.com
    • If you get a 403 Forbidden error, your bucket policy doesn't allow public reads
  • Versioning
    • Enabled at the bucket level
    • Same key overwrite will change the "version" of the object
    • Best practice to version your bucket to protect against unintended deletes. You can easily roll back to a previous version
    • Version ID is "null" if versioning was not enabled when object was created
    • Suspending versioning does not delete the previous versions
  • S3 Replication
    • Must enable versioning in source and destination buckets
    • Either Cross-Region Replication (CPR) or Same-Region Replication (SRR)
    • Buckets can be in different AWS accounts
    • Copying is asynchronous
    • Version IDs are replicated
  • S3 Encryption - Server-side by default, but can be client-side
  • IAM Access Analyzer for S3 helps ensure only intended people have access to your S3 buckets by analyzing bucket policies, ACLs and S3 Access Point Policies
  • AWS Storage Gateway - Use S3 data in a hybrid cloud environment by seamlessly bridging on-prem data and cloud data.
    • Use cases:
      • Disaster recovery
      • Backup & Restore
      • Tiered Storage
    • Types:
      • File Gateway
      • Volume Gateway
      • Tape Gateway

S3 Storage Classes

  • Durability - Represents how many times data will be lost by S3.
    • All S3 services have high durability (99.999999999%). If you store 10,000,000 objects with S3, you can expect to lose, on average, one object every 10,000 years. This is the same for all storage classes
  • Availability - Measures how readily available a service is. Varies depending on storage class
Storage Class NameNotesAvailabilityUse Cases
S3 Standard - General Purpose

No retrieval fee, no minimum storage duration, no minimum billable object size

99.999%

Big Data analytics, mobile & gaming applications, content distribution, etc

S3 Standard - Infrequent AccessMinimum billable object size of 128KB.99.9%Disaster Recovery, backups
S3 One Zone - Infrequent Access

Only stored in one AZ, rather than >= AZs. Min. billable object size of 128KB.

99.5%

Storing secondary backup copies of on-prem data, or data you can recreate

  • S3 Glacier Storage Classes - Low-cost object storage meant for archiving / backup. Pricing based on stage and object retrieval
Storage Class NameRetrieval TimeMinimum storage duration
S3 Glacier Instant RetrievalMillisecond retrieval, great for data accessed once a quarter90 days
S3 Glacier Flexible RetrievalExpedited (1-5mins), Standard (3-5hrs), Bulk (5-12hrs. Free!)90 days
S3 Glacier Deep ArchiveStandard (12hrs), Bulk (48 hrs)180 days
  • Also S3 Intelligent-Tiering
    • Tiers
      • Frequent Access Tier - Default
      • Infrequent Access Tier - Objects not accessed for 30 days
      • Archive Instant Access Tier - Objects not accessed for 90 days
      • Archive Access Tier - Optional. Configurable from 90 days to 700+ days
    • Small monthly monitoring and auto-tiering fee
    • Move objects automatically between access tiers based on usage
    • No retrieval charges
  • Can move between classes manually or using S3 Lifecycle configurations

S3 Shared Responsibility

Amazon:

  • Infrastructure (Global security, durability, availability, sustain concurrent loss of data in two facilities)
  • Configuration and vulnerability analysis
  • Compliance validation You:
  • S3 versioning
  • S3 Bucket Policies
  • S3 Replication Setup
  • Logging and Monitoring
  • S3 Storage Classes
  • Data encryption at rest and in transit

AWS Snow Family

  • The Snow Family is a family of highly secure, portable devices that are used to collect and process data at the edge and migrate data in and out of AWS.
  • Uses:
    • Data Migration - Write your data to a Snow device, ship it to an AWS facility where it will be imported or exported to your S3 bucket.
      • Motivation: Time to transfer extreme amounts of data over the internet can be prohibitively slow, costly and unstable. Snow Family can be used to perform offline data migrations. If it takes more than a week to transfer over the network, use Snowball devices!
    • Edge Computing is when you process data while it's being created on an edge location (somewhere with no internet or that is far away from a cloud, eg. truck on the road, ship on sea, mining station underground)
      • Use cases:
        • Preprocess data
        • Machine learning at the edge
        • Transcoding media streams

Snow Family usage process:

  1. Request Snowball devices from the AWS console
  2. Install the snowball client / AWS OpsHub on your servers
  3. Connect the snowball to your servers and copy files using the client
  4. Ship back the device when you're done
  5. Data loaded into the S3 bucket
  6. Snowball is completely wiped
  • AWS OpsHub - GUI interface for managing your Snow Family device:

    • Unlocking and configuring single/clustered devices
    • Transferring files
    • Launching and managing instances running on Snow Family Devices
    • Monitor device metrics
    • Launch compatible AWS services on your devices (eg EC2, AWS DataSync, NFS)
  • Snowball Edge - Use to move TBs or PBs of data in/out of AWS

    • Pay per data transfer job
    • Provide block storage and S3-compatible object storage
    • Types:
      • Storage Optimized
        • Up to 40 vCPUs, 80GiB of RAM, 80TB HDD storage
      • Compute Optimized
        • 104 vCPUs, 416GiB of RAM, 28TB NVMe or 42TB HDD usable storage
        • Optional GPU
        • Storage Clustering available (up to 16 nodes)
    • Use for large data cloud migrations, DC decommissioning or disaster recovery
  • Snowcone

    • Small, light, portable, rugged, withstands harsh environments
    • Used for edge computing, storage and data transfer
    • Must provide your own battery/cables
    • Send back to AWS offline or connect it to the internet and use AWS DataSync to send data
    • 2 CPUs, 4GB of memory, wired or wireless access
    • USB-C power using a cord or optional battery
    • Types:
      • Snowcone - 8TB of HDD capacity
      • Snowcone SSD - 14TB of SSD capacity
  • All of the above devices can run EC2 instances and Lambda functions using AWS IoT Greengrass

  • Long-term deployment options: 1 and 3 years (discounted)

  • Snowmobile

    • An actual truck.
    • Transfer exabytes of data (1EB = 1,000PB = 1,000,000TBs). Better than snowball if you want to send more than 10PBs of data
    • Each snowmobile has 100PB of capacity (use multiple in parallel)
    • High security, temperature controlled, GPS, 24/7 surveillance

Databases & Analytics

  • Use cases of databases:
    • Structured data storage
    • Build indexes to efficiently query the data
    • Define relationships between your datasets
  • Databases are optimized for a purpose and come with different features, shapes and constraints
  • Relational Databases - Tables, where each row has a unique key and values can be linked to other tables. Can use SQL to query
  • NoSQL / non-relational databases - Purpose-built databases that have flexible schemas for building modern apps
    • Benefits:
      • Flexible (easy to evolve model)
      • Scalability (designed to scale out by using distributed clusters)
      • High-performance (optimized for a specific data model)
      • Highly functional (types optimized for the data model)
  • Benefits of managing databases through AWS
    • Quick provisioning, high availability, vertical & horizontal scaling
    • Automated backup & restore, operations, upgrades
    • OS patching handled by AWS
    • Monitoring, alerting
  • Note: DB technologies that aren't officially supported by AWS can technically be run on EC2, but you'd need to manage everything yourself
  • Amazon RDS (Relational Database Service): Managed relational DB service.
    • Allows you to create DBs of type:
      • Postgres
      • MySQL
      • MariaDB
      • Oracle
      • MS SQL Server
      • IBM DB2
      • Aurora (AWS Proprietary DB) - AWS Implementation of PostgresQL / MySQL
        • Cloud optimized, claims 5x performance improvement over MySQL on RDS, 3x the performance of Postgres on RDS
        • Grows in increments of 10GB up to 128TB
        • Costs 20% more than RDS but is more efficient
        • Serverless option available
          • Automated database instantiation and auto-scaling based on actual usage
          • No capacity planning needed
          • Pay per second
          • Good for infrequent, intermittent or unpredictable workloads
    • RDS is a managed service:
      • Automated provisioning, OS patching
      • Continuous backups and restore specific timestamp
      • Monitoring dashboards
      • Read replicas for improved read performance
      • Multi AZ setup for DR
      • Maintenance windows for upgrades
      • Scaling capability
      • Storage backed by EBS
    • Cannot SSH into your instances
    • Architecture: ELB for multiple EC2 instances that all read/write to one DB
  • RDS Deployment options:
    • Read Replicas: Scale the read workload of your DB by creating up to 15 Read Replicas. Data is only written to the main DB
    • Multi-AZ: Cross-AZ read replication to one other AZ for failover in case of AZ outage. Data is only written to the main DB.
    • Multi-Region: Cross-Region read replication to other AZs for disaster recovery in case of region issue. Improves performance for global reads. Has a replication cost.
  • Amazon ElastiCache - Get a managed Redis or Memcached
    • Caches are in-memory DBs with high performance, low latency
    • Helps reduce load off DBs for read intensive workloads
    • AWS takes care of OS maintenance / patching, optimizations, setup, config, monitoring, failure recovery and backups
    • Architecture: ELB for EC2 Instances that can read/write from either cache or DB
  • DynamoDB - Fully managed highly-available with replication across 3 AZs.
    • NoSQL DB
    • Distributed serverless DB
    • Scales to massive workloads
    • Millions of requests per second, trillions of rows, 100TBs of storage
    • < 10ms latency
    • Integrated with IAM for security, authorization and administration
    • Low cost & auto-scaling capabilities
    • Standard or IA classes
    • Type of data:
      • Primary Key made up of:
        • Partition Key
        • Sort key (optional)
      • Attributes
        • Schema is defined per item
    • DynamoDB Accelerator (DAX) - Fully managed in-memory cache for DynamoDB.
      • 10x performance improvement
      • Fully integrated with DynamoDB
      • Secure, highly scalable & highly available
      • Different from ElastiCache, as DAX can only be used with DynamoDB
    • Global Tables - Make a DynamoDB table accessible with low latency in multiple regions.
      • Active-Active replication
      • Read/write to any AWS Region
  • Redshift - Online Analytical Processing (OLAP; for analytics and data warehousing) DB
    • Load data once every hour, not every second
    • 10x better performance than other data warehouses, scale to PBs of data
    • Columnar storage of data (instead of row based)
    • Massively Parallel Query Execution (MPP), highly available
    • Pay-as-you-go based on instances provisioned
    • Has an SQL interface for performing the queries
    • BI tools such as AWS Quicksight or Tableau integrate with it
    • Serverless Option Available
      • Automatically provisions and scales data warehouse underlying capacity
      • Run analytics workloads without managing data warehouse infrastructure
      • Pay for what you use (saves cost)
      • Use Cases: Reporting, dashboarding applications, real-time analytics
  • Amazon EMR (Elastic MapReduce) - Helps creating Hadoop clusters (Big Data) to analyze and process vast amounts of data.
    • Clusters can be made of hundreds of EC2 instances
    • Also supports Apache Spark, HBase, Presto, Flink
    • Takes care of all provisioning and configuration
    • Auto-scaling and integrated with Spot instances
    • Use Cases: data processing, machine learning, web indexing, big data
  • Amazon Athena - Serverless query service to analyze data stored in Amazon S3
    • Query with standard SQL
    • Supports CSV, JSON, ORC, Arvo and Parquet (built on Presto)
    • Pricing: $5.00 per TB of data scanned
    • Use Cases: Business intelligence / analytics / reporting, analyze & query VPC Flow Logs, ELB Logs, CloudTrail trails, etc
  • Amazon QuickSight - Serverless ML-powered BI service to create interactive dashboards
    • Fast, autoscalable, embeddable, with per-session pricing
    • Use Cases: Business analytics, building visualizations, perform ad-hoc analysis, get business insights using data
    • Integrated with RDS, Aurora, Athena, Redshift, S3...
  • DocumentDB - AWS implementation of MongoDB (NoSQL DB)
    • Use to store, query and index JSON data
    • Similar deployment concepts as Aurora
    • Fully Managed, highly available with replication across 3 AZs
    • Storage automatically grows in increments of 10GB
    • Automatically scales to workloads with millions of requests per second
  • Amazon Neptune - Fully managed graph DB
    • Highly available across 3 AZs with up to 15 read replicas
    • Optimized for complex and hard queries on highly connected datasets
    • Can store up to billions of relationships and query the graph with millisecond latency
    • Use cases: Knowledge graphs, fraud detection, recommendation engines, social networking
  • Amazon Timestream - Fully managed, fast, scalable, serverless time series DB (stores time-value pairs)
    • Automatically scales up/down to adjust capacity
    • Store and analyze trillions of events per day
    • 1000s of times faster and 1/10th the cost of relational DBs
    • Built-in time series analytics functions to help you identify patterns in your data in near real-time.
  • Amazon QLDB (Quantum Ledger Database) - Fully managed, serverless transaction recording
    • Highly available, replication across 3 AZs
  • Amazon Managed Blockchain - Managed service for joining public blockchain networks or creating your own scalable private network
    • Compatible with Hyperledger Fabric & Ethereum
  • AWS Glue - Managed, fully serverless extract, transform and load (ETL) service
    • Useful to prepare and transform data for analytics
    • Can extract from multiple services, send to Redshift for example
  • Glue Data Catalog - Catalog of datasets in your AWS infrastructure
    • Can be used by Athena, Redshift, EMR
  • DMS (Database Migration Service) - QUickly and securely migrate DBs to AWS, resilient, self-healing
    • Source DB still available
    • Supports migrations to DBs of same or other type

Other Compute Services

ECS (Elastic Container Service), Fargate & ECR

  • Docker - Software development platform to deploy apps packaged in containers so they'll run the same on any machine
    • Benefits:
      • No compatibility issues
      • Predictable behaviour
      • Less work
      • Easier to maintain and deploy
      • Works with any language, OS or technology
      • Scales up and down very quickly
    • Docker Daemon runs on host OS, containers are then run on Docker Daemon
  • Docker images are stored in Docker Repositories:
    • Public: Docker Hub
    • Private: Amazon ECR (Elastic Container Registry)
  • Amazon ECS (Elastic Container Service) - Launch Docker containers on AWS
    • You must provision & maintain the infrastructure (the EC2 instances)
    • AWS takes care of starting / stopping containers
    • Integrates with the Application Load Balancer
  • Fargate - Serverless service to launch Docker containers on AWS
    • Runs containers for you based on the CPU/RAM you need
  • ECR and Fargate can run containers stored on ECR

Serverless

  • Serverless - Deploying code without the need to manage servers
    • Pioneered by Lambda but now also includes databases, messaging, storage, etc
    • Includes S3, DynamoDB, Fargate, Lambda
  • AWS Lambda - Virtual functions run on-demand with automated scaling
    • Motivation: EC2 instances are limited by RAM & CPU, are continuously running, scaling requires intervention to add/remove servers
    • Benefits:
      • Pay per request and compute time.
        • Free tier of 1,000,000 AWS Lambda requests and 400,000GB of compute time
        • $0.20 per 1,000,000 requests after
        • $1.00 for 600,000 GB-seconds after
      • Integrate with whole AWS suite of services
      • Event-Driven - funcitons get invoked by AWS when needed
      • Integrated with many programming languages
      • Easy monitoring through AWS CloudWatch
      • Easy to get more resources per functions (up to 10GB of RAM)
    • Supported Languages
      • Node.js
      • Python
      • Java
      • C# (.NET Core)
      • Golang
      • C# / Powershell
      • Ruby
      • Any language through the custom runtime API
    • Events are sent into lambdas as JSON object arguments
  • Amazon API Gateway - Fully managed service for devs to easily create, publish, maintain, monitor and secure APIs
    • Serverless and scalable
    • RESTful or WebSocket APIs

Batch, Lightsail

  • AWS Batch - Fully managed batch processing at any scale
    • Batch jobs are jobs with a 'start' and an 'end', as opposed to continuous jobs
    • Dynamically launches EC2 or Spot instances, provisioned with the right amount of compute/memory
  • Amazon Lightsail - Virtual servers, storage, DBs, networking
    • Low & predictable pricing
    • Great for people with little cloud experience
    • Can setup notifications and monitoring of your Lightsail resources
    • Use cases - Simple webapps (LAMP, Nginx, MEAN, Node...), websites (WordPress, Megento, Plesk, Joomla...), dev/test environment
    • High availability, no auto-scaling, limited AWS integrations

Deployments & Managing Infrastructure at Scale

  • CloudFormation - Declarative way of outlining your AWS infrastructure. Supports most resources.
    • Provide a template and CloudFormation will create specified services in the right order with the exact configuration you specify.
    • Use Cases - Repeat architecture in different environments, regions or accounts.
    • Benefits:
      • Infrastructure as Code - No manual creation of resources, which is excellent for control. Changes to the infrastructure are reviewed as code
      • Cost - Each resource within the stack is tagged with an identifier so you can easily see how much a stack costs you. You can estimate the costs of your resources using the CloudFormation template. You could automate regular safe deletion and recreation of templates when needed to save costs.
      • Productivity - Destroy and recreate infrastructure on the cloud on the fly. Automated diagram generation.
      • Avoid re-inventing the wheel - Leverage existing templates on the web and their documentation
  • AWS Cloud Development Kit (CDK) - Define your cloud infrastructure using a familiar language (JS/TS, Python, Java, .NET) to be compiled into a CloudFormation template (JSON/YAML)
  • AWS Elastic Beanstalk - Developer-centric view of deploying an application on AWS. Uses EC2, ASG, ELB, RDS, etc in one view that's easy to make sense of, with you still in control of the configuration.
    • Platform as a Service (PaaS)
    • Manages:
      • Instance / OS configuration
      • Deployment strategy
      • Capacity provisioning
      • Load balancing, auto-scaling
      • Application health monitoring and responsiveness
    • Three architecture models:
      • Single instance (use for dev)
      • LB + ASG (use for production/pre-production)
      • ASG only (use for non-web apps in production)
  • AWS CodeDeploy - Hybrid service for automatic application deployment
    • Works with EC2 instances or on-prem servers (must be provisioned and configured ahead of time with the CodeDeploy Agent)
  • AWS CodeCommit - Git-based repository within AWS
  • AWS CodeBuild - Code building in the cloud
    • Compiles source code, run tests and produces packages that are ready to be deployed (eg.by CodeDeploy)
    • Benefits:
      • Fully managed, serverless
      • Continuously scalable & highly available
      • Secure
      • Pay-as-you-go
  • AWS CodePipeline - Orchestrate the different steps to have code automatically pushed to production (CI/CD)
    • Benefits:
      • Fully managed, compatible with CodeCommit, CodeBuild, CodeDeploy, Elastic Beanstalk, CloudFormation, GitHub, third-party services
      • Fast delivery & rapid updates
  • AWS CodeArtifact - Secure, scalable artifact management service for software development
    • Artifact Management is the storing and retrieval of dependencies
    • Works with common dependency management tools like Maven, Gradle, npm, yarn, twine, pip and NuGet
    • Compatible with CodeBuild
  • AWS CodeStar - Unified UI to manage software dev activities in one place
  • AWS Cloud9 - Cloud IDE for writing, running and debugging code in the cloud.
  • AWS Systems Manager (SSM) - Hybrid service to manage your EC2 and on-prem systems at scale
    • Important Features:
      • Patching automation for enhanced compliance
      • Run commands across an entire fleet of servers
      • Store parameter configuration with the SSM Parameter Store
    • Works on Linux, Windows, MacOS and Raspbian (must have SSM agent installed)
    • SSM Session Manager - Allows you to start a secure shell on an instance/server with the SSM Agent installed
      • No SSH access, bastion hosts, SSH keys or port 22 needed (better security)
      • Session log data can be sent ot S3 or CloudWatch logs
    • SSM Manager Parameter Store - Secure storage for configuration and secrets
      • Serverless, scalable, durable, easy SDK
      • Control authorization using IAM
      • Version tracking & encryption (optional)

Leveraging the AWS Global Infrastructure

  • Global Applications are applications deployed in multiple geographies (regions and/or edge locations)
    • Decreased Latency
    • Disaster Recover
    • Attack protection
  • Amazon Route 53 - Managed DNS (collection of rules and records which helps clients understand how to reach a server through URLs)
    • Routing Policies
      • Simple Routing Policy - No health checks
      • Weighted Routing Policy - Health checks, ensure certain percentage of the load goes to certain instances
      • Latency Routing Policy - Route to minimize user latency
      • Failover Routing Policy - Perform health check on primary instance, and redirect to failover instance if primary is not healthy
    • Costs
      • $12 a year for the domain
      • $0.50 per month for the hosted zone
  • CloudFront - CDN that improves read performance by caching content at edge locations (216 PoPs available)
    • Provides DDoS protection, integration with Shield, AWS Web Application Firewall
    • Origins - S3 buckets or custom
    • Files are cached for a specified Time-to-Live
    • Great for static content that must be available everywhere
  • S3 Transfer Acceleration - Increase transfer speed by transferring file to an edge location which will forward the data to the S3 bucket in the target region
  • AWS Global Accelerator - Improve global application availability and performance using the AWS global network, leveraging the AWS internal network to optimize the route to your application.
    • Use Cases - HTTP services that require static IP addresses and/or deterministic, fast regional failover
  • AWS Outposts - Server racks that offer the same AWS infrastructure, services, API and tools to build applications on-prem just as you could in the cloud.
    • You are responsible for the Outpost's physical security
    • Benefits
      • Low-latency access to on-prem systems
      • Local data processing
      • Data residency
      • Easier migration from on-prem to cloud
      • Fully managed service
  • AWS Wavelength - Ultra-low latency applications by deploying to telcom providers' datacenters at the edge of the 5G network.
    • Traffic doesn't leave the Communication Service Provider's (CSP) network
    • High-bandwidth and secure connection to the parent AWS region
    • No additional charges or service agreements
    • Use cases - Anything that requires very low latency
  • AWS Local Zones - Place AWS compute, storage, DB and other selected AWS services closer to end users to run latency-sensitive applications
    • Extend your VPC to more locations

Cloud Integrations

  • Amazon SQS (Standard Queue Service) - Producers send messages to the queue, consumers poll the queue to request messages.
    • Standard Queue - Fully managed, used to decouple applications
      • Scales from 1 message per second to tens of thousands per second
      • Default retention of messages: 4 days, maximum of 14 days
      • No limit to how many messages can be in the queue
      • Low latency (<10ms on publish and receive)
      • Consumers share the work to read messages and scale horizontally
    • FIFO Queue - First In, First Out Queue. Messages are processed in order by the consumer
  • Amazon Kinesis - Managed service to collect, process and analyze real-time streaming data at any scale
    • Kinesis Data Streams - Low latency streaming to ingest data at scale from hundreds of thousands of sources
    • Kinesis Data Firehose - Load streams into S3, Redshift, ElasticSearch, etc
    • Kinesis Data Analytics - Perform real-time analytics on streams using SQL
    • Kinesis Video Streams - Monitor real-time video streams for analytics or ML
  • Amazon SNS (Simple Notification Service) - Pub/sub messaging service
    • "Event publishers" only send messages to one SNS topic
    • Many "event subscribers" as we want to listen to the SNS topic notifications
    • Each subscriber to the topic will get all the messages
    • Limit of 12,500,000 subscriptions per topic, 100,000 topics
  • Amazon MQ - Managed message broker service for RabbitMQ and ActiveMQ, rather than using the cloud-native proprietary protocols used by SQS and SNS.
    • Doesn't scale as much as SQS, SNS.
    • Runs on multiple servers, can run in Multi-AZ with failover
    • Has both queue features and topic features
    • Use case if migrating to the cloud and want to avoid re-engineering the application to use SQS/SNS

Cloud Monitoring

  • Amazon CloudWatch Metrics - Provides metrics for every service in AWS
    • Metrics - Variables to monitor (CPU utilization, network usage, etc)
    • Important Metrics
      • For EC2 instances, CPU util, status checks, network (not RAM). Default every 5 minutes, can pay extra to generate every 1 minute
      • For EBS volumes, disk read/writes
      • For S3 buckets, BucketSizeBytes, NumberOfObjects, AllRequests
      • For billing, total estimated charge (only in us-east-1)
      • For service limits, how much you've been using a service API
      • Or push your own custom metrics
    • Metrics are timestamped
    • Can create CloudWatch dashboard of metrics
  • CloudWatch Alarms - Once a metric passes a specified threshold, triggers a notification
    • Alarm actions include
      • Auto Scaling
      • EC2 actions
      • SNS notifications
    • Various options for threshold, including sampling, percentage, max, min, etc..
    • Can choose the period on which to evaluate an alarm
  • CloudWatch Logs - Real-time monitoring of logs from AWS services
    • Can collect logs from:
      • Elastic Beanstalk (from application)
      • ECS (from containers)
      • AWS Lambda (from function logs)
      • CloudTrail (based on filter)
      • CloudWatch log agents (on EC2 instances or on-prem servers)
      • Route53 (DNS queries)
    • Can adjust retention
    • Must make sure IAM permissions are correct for logs to be pushed through the CloudWatch log agent
  • Amazon EventBridge
    • Perform an action based on:
      • A schedule (cron jobs)
      • An event pattern (react to a service doing something)
    • Formerly CloudWatch Events
  • AWS CloudTrail - Provides governance, compliance and auditing for your AWS account
    • Enabled by default
    • Get a history of events/API calls made within your AWS account by:
      • Console
      • SDK
      • CLI
      • AWS Services
    • Can put logs from CloudTrail into CloudWatch Logs or S3
    • A trail can be applied to all regions (default) or a single region
  • AWS X-Ray - Visual analysis of your applications for debugging and troubleshooting
    • Advantages:
      • Troubleshooting performance bottlenecks
      • Understand dependencies in a microservice architecture
      • Pinpoint service issues
      • Review request behaviour
      • Find errors and exceptions
      • See where you are being throttled
      • Identify users that are impacted
  • Amazon CodeGuru - ML-powered service for automated code reviews and application performance recommendations
    • CodeGuru Reviewer - Automated code reviews for static code analysis
      • Trained on thousands of open-source and Amazon repositories
      • Supports Java and Python
      • Integrates with GitHub, Bitbucket and CodeCommit
    • CodeGuru Profiler - Recommendations about application performance during runtime
      • Identifies and removes code inefficiencies
      • Decrease compute costs
      • Provides heap summary (identify which objects are using up memory)
      • Anomaly detection
      • Supports both cloud and on-prem application
      • Minimal overhead on application
  • AWS Health Dashboard
    • Service History - Shows health for all services in all regions for each day. Has an RSS feed you can subscribe to
    • Health Dashboard - Gives you a personalized view into the performance and availability of the AWS services underlying your AWS resources. Displays relevant and timely information to help you manage events in progress and provides proactive notification to help you plan for scheduled activities

VPC and Networking

  • IP Addresses in AWS
    • EC2 instances get new public IP addresses every time you stop and start them
    • Private IPv4 is fixed for EC2 instances even if you start/stop them
    • Elastic IP - Allows you to attack a fixed public IPv4 address to an EC2 instance
    • NOTE: All public IPv4 on AWS will be charged at $0.005 per hour
  • VPC (Virtual Private Cloud) - Private network to deploy your resources (regional resource)
    • Subnets allow you to partition your network inside your VPC (AZ resource)
    • A public subnet is a subnet that is accessible from the internet
    • To define access to the internet and between subnets, we use Route Tables
    • Internet Gateways allow VPC instances connect with the internet
    • NAT Gateways (AWS-managed) & NAT Instances (self-managed) allow your instances in your Private Subnets to access the internet while remaining private
  • NACL (Network ACL) - Firewall which controls traffic from and to subnet. Rules only include IP addresses, supports ALLOW and DENY
  • VPC Flow Logs - Captures information about IP traffic going into your interfaces (VPC, Subnet, Elastic Network Interface logs)
    • Helps to monitor & troubleshoot connectivity issues (subnets to internet, internet to subnets or subnets to subnets)
    • Also captures network information from AWS managed interfaces (eg ELB, ElastiCache, RDS, Aurora)
    • Flow data can go to S3, CloudWatch Logs and Kinesis Data Firehose
  • VPC Peering - Connect two VPCs privately, making them behave as if they were in the same network
    • VPCs must not have overlapping CIDR (IP address range)
    • VPC Peering connection is not transitive
  • VPC Endpoints allow you to connect to AWS services using a private network instead of the internet, giving you enhanced security and lower latency to access AWS services
  • AWS PrivateLink (VPC Endpoint Services) - Most secure & scalable way to expose a service to 1000s of VPCs, requiring only a network load balancer (Service VPC) and ENI (Customer VPC)
  • Site-to-site VPN - Automatically encrypted connection between an on-prem VPN and AWS over the public internet
    • On-prem must use a Customer Gateway (CGW)
    • AWS must use a Virtual Private Gateway (VGW)
  • Direct Connect (DX) - Establish a physical connection between on-prem and AWS over a private network.
    • Takes at least a month to establish
  • AWS Client VPN - Connect from your computer using OpenVPN to your private network over the public internet
    • Allows you to connect to your EC2 instances over a private IP as if you were in the private VPC network
  • Transit Gateway - For transitive peering between many clients via one single gateway. Hub-and-spoke/Star connection
    • Works with Direct Connect Gateway, VPN connections

Security and Compliance

  • DDoS protection on AWS
    • AWS Shield - DDoS protection for your website and applications
      • AWS Shield Standard is enabled by default for all customers at no additional cost
        • Provides protection from attacks such as SYN/UDP floors, reflection attacks and other layer 3/layer 4 attacks
      • AWS Shield Advanced - 24/7 premium DDoS protection
        • $3,000 per month per organization
        • Protect against more sophisticated attacks on EC2, ELBs, CloudFront, Global Accelerator and Route53
        • 24/7 access to AWS DDoS Response Team (DRP)
    • AWS WAF (Web Application Firewall) - Filter specific requests based on rules
      • Protects your applications from common layer 7 (HTTP-level) exploits
      • Deploy on Application Load Balancer, API Gateway, CloudFront
      • Define Web ACLs - Rules can include IP addresses, HTTP headers & bodies, URI strings, source county, rate of requests
    • CloudFront and Route53 provide availability protection using global edge network
    • Leverage AWS Auto Scaling
  • AWS Network Firewall - Protect your entire VPC, inspecting in any direction
  • AWS Firewall Manager - Manage security rules in all accounts of an AWS org. Rules will be applied to all new resources as they are created (good for compliance)
  • Penetration Testing on AWS Cloud
    • AWS Customers can carry out security assessments/pentests against their AWS infrastructure without prior approval on 8 services:
      • EC2 instances, NAT Gateways, ELBs
      • RDS
      • CloudFront
      • Aurora
      • API Gateways
      • Lambda and Lambda Edge functions
      • Lightsail resources
      • Elastic Beanstalk environments
    • Prohibited Activities
      • DNS zone walking via Amazon Route 53 Hosted Zones
      • DoS, DDoS or simulated versions of either
      • Port flooding
      • Protocol flooding
      • Request flooding
  • AWS KMS (Key Management Service) - Management of encryption keys for services that encrypt data (default for CloudTrail, S3 Glaciers, Storage Gateway. Opt-in for EBS volumes, S3 buckets, RedShift DBs, RDS DBs, EFS drives)
    • Types of keys:
      • Customer Managed
      • AWS Managed
      • AWS Owned
      • CloudHSM Keys
  • CloudHSM (Hardware Security Module) - Dedicated hardware module for managing your own encryption keys. Tamper resistant
  • AWS Certificate Manager - Lets you easily provision, manage and deploy SSL/TLS Certificates to provide in-flight encryption for websites (HTTPS)
    • Supports both public and private TLS certs
    • Free of charge for public TLS certs
    • Automatic TLS cert renewal
  • AWS Secrets Manager - Newer service for storing and managing secrets
    • Automate generation of secrets on rotation using Lambda
    • Secrets encrypted using KMS
    • Mainly meant for RDS integration
  • AWS Artifact - Portal that provides customers with on-demand access to AWS compliance docs and AWS agreements. Can be used to support internal audit or compliance
  • Amazon GuardDuty - ML-powered anomaly detection / threat discovery to protect your AWS account
    • Input includes:
      • CloudTrail Event Logs (incl. management events, s3 data events)
      • VPC Flow logs
      • DNS Logs
      • Other opt-in logs
    • Can setup EventBridge rules to be notified in case of findings (bridge to Lambda or SNS)
  • Amazon Inspector - Automated security assessments
    • Assesses:
      • EC2 instances:
        • Leveraging the SSM agent
        • Analyze against unintended network accessibility
        • Analyze the running OS against known CVEs
      • For container images pushed to ECR:
        • Assessment of container images as they are pushed using database of CVEs
      • For Lambda Functions
        • Identifies software vulns in function code and package dependencies
        • Assessment of functions as they are deployed
      • Risk score is associated with all vulns for prioritization
  • AWS Config - Helps with auditing and recording compliance of your AWS resources by recording configurations
    • Can store the configuration data into S3
    • Use cases: To answer:
      • Is there unrestricted SSH access to my security groups?
      • Do my buckets have any public access?
      • How has my ALB configuration changed over time?
    • You can receive alerts (SNS notifications) for any changes
    • AWS Config is a per-region service, but can be aggregated across regions and accounts
  • AWS Macie is a data security and privacy service that uses ML and pattern matching to identify sensitive data (such as personally identifiable information (PII)) in AWS
  • AWS Security Hub - Central security tool to manage security across several AWS accounts and automate security checks
    • Integrated dashboards showing current security and compliance status
    • Automatically aggregates alerts in predefined or personal findings formats from the other tools discussed in this section & AWS partner tools
    • Must first enable the AWS Config Service
  • Amazon Detective - ML- and graph-powered tool to analyze, investigate and quickly identify the root cause of security issues or suspicious activities
    • Automatically collects and processes events from VPC Flow logs, CloudTrail, GuardDuty and creates a unified view
    • Produces visualizations with details and context to get to the root cause
  • Abusive & prohibited behaviours on AWS:
    • Spam
    • Port scanning
    • DoS or DDoS
    • Intrusion attempts
    • Hosting objectionable (illegal) or copyrighted content
    • Distributing malware
  • IAM Access Analyzer - Find which resources can be accessed outside of the Zone of Trust

Machine Learning

  • Amazon Rekognition - Find objects, people, text, scenes in images and videos using ML
    • Facial analysis and facial search to do user verification, people counting
    • Create a database of 'familiar faces' or compare against celebrities
  • Amazon Transcribe - Automatically convert speech to text
    • Uses a deep learning process called Automatic Speech Recognition (ASR)
    • Can automatically remove PII
    • Supports Automatic Language Identification for multi-lingual audio
    • Use cases
      • Transcribe customer calls
      • Automate closed captioning and subtitling
      • Generate metadata for media assets to create a fully searchable archive
  • Amazon Polly - Turn text into lifelike speech using deep learning
  • Amazon Translate - Natural and accurate language translation
  • Amazon Lex - ASR to convert speech to text. Powers Alexa
  • Amazon Connect - Receive calls, create contact flows, cloud-based virtual contact center
    • Can integrate with other CRM systems of AWS
    • No upfront payments, 80% cheaper than traditional contact center solutions
    • Integrate with Lex to recognize the intent of callers
  • Amazon Comprehend - Fully managed and serverless service for Natural Language Processing
    • Uses ML for
      • Language recognition
      • Tokenization
      • Topic extraction
      • Sentiment analysis
    • Sample use cases
      • Analyze customer emails to find what leads to a positive or negative experience
      • Create and group articles by topics that Comprehend will uncover
  • Amazon SageMaker - Fully managed service for developers / data scientists to build ML Models
  • Amazon Forecast - Fully managed service that uses ML to deliver highly accurate forecasts based on uploaded time-series data
    • Uses cases
      • Product Demand Planning
      • Financial Planning
      • Resource Planning
  • Amazon Kendra - Fully managed document search service powered by ML
    • Extract answers to questions from within an input document
    • Learn from user interactions/feedback to promote preferred results (incremental learning)
  • Amazon Personalize - Fully managed ML-service to build apps with real-time personalized recommendations
    • Same technology used by Amazon.com
    • Integrates into existing websites, applications, SMS, email marketing systems, etcA
    • Use cases
      • Retail stores
      • Media and entertainment services
  • Amazon Textract - Automatically extracts text, handwriting and data from any scanned documents using AI and ML
    • Reads and processes any type of document (PDFs, images, etc) and extracts data
    • Use cases
      • Financial services (eg. invoices, financial reports)
      • Healthcare (eg medical records, insurance claims)
      • Public Sector (eg tax forms, ID documents, passports)

Account Management, Billing & Support

Account Management

  • AWS Organizations - Global service allowing management of multiple AWS accounts
    • Cost Benefits
      • Consolidated billing across all accounts (single payment method)
      • Pricing benefits from aggregated usage (volume discount for EC2, S3, etc)
      • Pooling of Reserved EC2 instances for optimal savings
    • API is available to automate AWS account creation
    • Restrict account privileges using Service Control Policies (SCP)
  • Multi-Account Strategies - Create accounts in Organizational Units, such as business (sales, retail, finance), environment (production, development, test) or project
  • Service Control Policies - Whitelist or blacklist IAM actions at the OU or Account level (does not apply to the Master/main account)
    • Does not affect service-linked roles
    • Must have an explicit Allow (doesn't allow anything by default)
    • Use cases
      • Restrict access to certain services
      • Enforce PCI compliance by explicitly disabling services
  • AWS Control Tower - Easy way to set up and govern a secure and compliance multi-account AWS environment based on best practices
    • Benefits
      • Automate ongoing policy management using guardrails
      • Detect policy violations and remediate them
      • Monitor compliance through an interactive dashboard
    • Runs on top of AWS Organizations
  • AWS Resource Access Manager (AWS RAM) - Share AWS resources you own with other AWS accounts to avoid resource duplication
    • Supported resources include Aurora, VPC subnets, Transit Gateway, Route 53, EC2 Dedicated Hosts, License Manager Configurations
  • AWS Service Catalog - Quick self-service portal to launch a set of authorized products pre-defined by org admins
    • Avoids users that are new to AWS creating stacks that are not compliant with standards or in-line with existing architecture

Pricing

  • Pricing Models
    • Pay as you go
    • Save when you reserve
    • Pay less by using more (volume-based discounts)
    • Pay less as AWS grows
  • Free services & free tier in AWS
    • IAM
    • VPC
    • Consolidated Billing
    • EC2 t2.micro instance (first year)
    • S3, EBS, ELB, AWS Data transfer
    • Free services where you pay for the resources created:
      • Elastic Beanstalk
      • CloudFormation
      • Auto Scaling Groups
  • EC2
    • Pay as you go
    • On-demand instances: Pay per second (linux/windows) or per hour (other)
    • Reserved instances:
      • Up to 75% discount compared to On-demand hourly rate
      • 1- or 3-years commitment
      • All upfront, partial upfront, no upfront
    • Spot instances:
      • Up to 90% discount compared to on-demand hourly rate
      • Bid for unused capacity
    • Dedicated Host
      • On-demand pricing
      • Reservation for 1- or 3-years commitment
    • Savings plan to save on sustained usage
  • Lambda - Pay per call, pay per duration
  • ECS - Pay for AWS resources stored and created in your application
  • Fargate - Pay for vCPU and memory resources allocated to your applications in your containers
  • S3 - Pay depends on:
    • Storage class
    • Number and size of objects (price can be tiered based on volume)
    • Number and type of requests
    • Data transfer out of the S3 region
    • S3 Transfer acceleration
    • Lifecycle transitions
  • EBS - Pricing depends on
    • Volume type (based on performance)
    • Storage volume in GB per month provisioned
    • IOPS
      • Included when using General Purpose SSD
      • Pay for provisioned amount when using Provisioned IOPS SSDs
    • Pay per number of requests when using Magnetic
    • Snapshots - Added data cost per GB per month
    • Data transfer - Outbound data transfer tiered for volume discounts
  • RDS - Per hour billing, depending on:
    • Database characteristics (engine, size, memory class)
    • Purchase type (on-demand or reserved)
    • Additional storage (per GB per month)
    • Number of input and output requests per month
    • Deployment type (storage and I/O are variable)
    • Outbound data transfer
    • NOTE - Inbound data transfer and backup storage is free
  • CloudFront - Different across different geographic regions. Pricing is aggregated for each edge location and then applied to your bill, depending on:
    • Data transfer out (volume discount)
    • Number of HTTP/HTTPS requests
  • Networking Costs
    • Traffic in is free
    • Traffic between instances in the same AZ is free if using private IP
    • Traffic between instances in different AZs is either $0.02 per GB if using public/elastic IP to communicate or $0.01 per GB if using private IP to communicate
    • Traffic between regions is $0.02 per GB
  • Savings Plan - Commit to a certain dollar amount per hour for 1 or 3 years
    • EC2 Savings plan - Up to 72% discount compared to on-demand
      • Commit to usage of individual instance families in a region, regardless of AZ, size, OS or tenancy
      • All upfront, partial upfront or no upfront
    • Compute Savings Plan
      • Up to 66% discount compared to On-Demand
      • Regardless of family, region, size, OS, tenancy, compute options used
    • Machine Learning Savings Plan for ML services
    • Set up form the AWS Cost Explorer Console
  • AWS Compute Optimizer - Reduce costs and improve performance by recommending optimal AWS resources for your workloads
    • ML-powered, analyzes your resources' configuration and their utilization CloudWatch metrics
    • Supports:
      • EC2 instances,
      • EC2 ASGs
      • EBS volumes
      • Lambda functions
    • Can help lower your costs by up to 25%
    • Recommendations can be exported to S3

Billing and Costing

  • AWS Pricing Calculator - Estimate the cost for your solution architecture
  • AWS Billing Dashboard - Shows you all the cost forecast and the month-to-date cost
  • AWS Free Tier Dashboard - Shows you the Free Tier usage limits, how close you are to reaching the limit for that month and how close you'll be to reaching the limit for that month (forecasted)
  • Cost Allocation Tags - Used to track your AWS costs on a detailed level
    • AWS-generated tags:
      • Automatically applied to the resource you create
      • Starts with the Prefix aws:
    • User-defined tags
      • Starts with Prefix user:
    • Use tags to create resource groups so you can view the total cost for that group
  • Cost and Usage Reports - Produces comprehensive reports on AWS cost and usage data (including additional metadata about AWS services, pricing and reservations)
    • Lists AWS usage for each service category used by an account and its IAM users in hourly or daily line items, as well as any tags that you have activated for cost allocation purposes.
    • Can be integrated with Athena, Redshift or QuickSight
  • Cost Explorer - Creates custom reports analyzing cost and usage data to help you visualize, understand and manage your AWS costs and usage over time.
    • Analyze cost data by account, month, hour or resource
    • Choose an optimal Savings Plan
    • Forecast usage up to 12 months based on previous usage
  • Billing data can be used as a metric to trigger CloudWatch
  • AWS Budgets - Create a budget and send alarms when costs exceed the budget limit
    • 4 budget types: Usage, Cost, Reservation, Savings Plans
    • Track utilization of Reserved Instances (for EC2, ElastiCache, RDS, Redshift)
    • 5 SNS notifications per budget
    • Can filter by service, linked account, tag, purchase option, instance type, region, availability zone, API operation, etc
    • 2 budgets are free, the $0.02 per day per budget
  • AWS Cost Anomaly Detection - ML-powered detection mechanism that learns your unique, historic spend patterns to detect unusual cost spikes and/or continuous cost increases
    • Monitor AWS services, member accounts, cost allocation tags or cost categories
    • Sends you the anomaly detection report with root-cause analysis
    • Get notified with individual alerts or with a daily/weekly summary (using SNS)
  • AWS Service Quotas - Notifies you when you're close to a service quota value threshold
    • Create CloudWatch Alarms in the Service Quotas console

Support

  • Trusted Advisor - High-level AWS account assessment
    • Analyze your AWS accounts and provides recommendations for 6 categories:
      • Cost optimization
      • Performance
      • Security
      • Fault tolerance
      • Service limits
      • Operational excellence
    • Business & Enterprise Support plan
      • Full set of checks
      • Programmatic Access using AWS Support API
  • AWS Support Plans Pricing:
    • Basic: Free
    • Developer - Greater of $29.00 or 3% of monthly AWS charges
    • Business - Greater of:
      • $100.00
      • 10% of monthly AWS charges for the first $0-10k
      • 7% of monthly AWS charges from $10-80k
      • 5% of monthly AWS charges from $80-250k
      • 3% of monthly AWS charges over $250k
    • Enterprise On-Ramp - Greater of $5,500 or 10% of monthly AWS charges
    • Enterprise - Greater of:
      • $15,000
      • 10% of monthly AWS charges for the first $0-150k
      • 7% of monthly AWS charges from $150-500k
      • 5% of monthly AWS charges from $500k-1M
      • 3% of monthly AWS charges over $1M
  • AWS Support Plans
    • Basic
      • 24/7 access to customer service, documentation, whitepapers and support forums
      • Access to the 7 core Trusted Advisor checks and guidance to provision your resources following best practices to increase performance and improve security
      • Access to AWS Personal Health Dashboard
    • Developer
      • Basic support plan features +
      • Business hours email access to CloudSupport Associates
      • Unlimited cases / 1 primary contact
      • Case severity / response times:
        • General guidance - less than 24 business hours
        • System impaired - less than 12 business hours
    • Business
      • Intended to be used if you have production workloads
      • Access to the full suite of checks and API calls of Trusted Advisor
      • 24/7 phone, email and chart access to Cloud Support Engineers
      • Unlimited cases and contacts
      • Access to Infrastructure Event Management for an additional fee
      • Case severity / response times:
        • Same as Developer for general / system impaired
        • Production system impaired - less than 4 hours
        • Production system down - less than 1 hour
      • Enterprise On-Ramp
        • Intended to be used if you have production or business critical workloads
        • Same as Business Plan, and..
        • Access to a pool of Technical Account Managers (TAMs)
        • Concierge Support Team (for billing and account best practices)
        • Infrastructure Event Management, Well-Architected & Operations Reviews
        • Case severity / response times:
          • Same as business for general, system impaired, prod system impaired, prod system down
          • Business-critical system down - less than 30 minutes
      • Enterprise
        • Same as Enterprise On-Ramp and...
        • Access to a designated Technical Account Manager
        • Case severity / response times:
          • Same as enterprise on-ramp, but..
          • Business-critical system down - less than 15 minutes

Advanced Identity

  • AWS STS (Security Token Service) enables you to create temporary, limited privilege credentials to access your AWS resources
    • Creates short-term credentials where you configure the expiration period
    • Use cases
      • Identity federation (manage user identities in external systems and provide them with STS tokens to access AWS resources)
      • IAM roles for cross/same account access
      • IAM roles for Amazon EC2 (provide temp credentials for EC2 instances to access AWS resources)
  • Amazon Cognito - ALlows users of your application to login via SSO providers (Google, Facebook, etc) instead of creating them an IAM user
  • AWS Directory Services
    • Microsoft Active Directory - Database of objects stored on Windows Server instances with AD Domain Services storing information about user accounts, computers, printers, file shares and security groups. Creates a centralized store for security management, account creation and permission assignment
    • AWS Managed Microsoft AD - Create your own AD in AWS, manage users locally. Establish 'trust' connections with your on-prem AD. Supports MFA.
    • AD Connector - Proxy Directory Gateway that redirects to your on-prem AD. Users are managed on the on-prem AD. Supports MFA
    • Simple AD - AD-compatible managed directory on AWS. Cannot be joined with on-prem AD
  • AWS IAM Identity Center
    • SSO for:
      • AWS accounts in AWS organizations
      • Business cloud applications
      • SAML2.0-enabled applications
      • EC2 Windows Instances
    • Identity providers:
      • Built-in identity store in IAM Identity Center
      • 3rd party: AD, OneLogin, Okta...

Other AWS Services

  • Amazon WorkSpaces - Managed Desktop as a Service (DaaS) solution to easily provision Windows or Linux desktops
    • Eliminates management of on-prem VDI (Virtual Desktop Infrastructure)
    • Fast and quickly scalable to thousands of users
    • Secured data (integrates with KMS)
    • Pay-as-you-go with monthly or hourly rates
  • Amazon AppStream 2.0 - Desktop application streaming service
    • Deliver to any computer without the need to acquire or provision infrastructure
    • Delivered from within a web browser
  • AWS IoT Core - Allows you to easily connect IoT devices to the AWS Cloud
    • Serverless, secure, scalable to billions of devices, trillions of messages
    • Your apps can communicate with your devices even when they aren't connected
    • Integrates with many AWS services (Lambda, S3, SageMaker)
  • AWS AppSync - Store and sync data across mobile and web apps in real-time
    • Makes use of GraphQL
    • Client code can be generated automatically
    • Integrations with DynamoDB / Lambda
    • Real-time subscriptions
    • Offline data sync (replaced 'Cognito Sync')
    • Fine-grained security
    • AWS Amplify can leverage AppSync in the background
  • AWS Amplify - Set of tools and services that helps you develop and deploy scalable full-stack web and mobile apps
    • Integrates:
      • Authentication
      • Storage
      • API (REST, GraphQL)
      • CI/CD
      • PubSub
      • Analytics
      • AI/ML Predictions
      • Monitoring
      • Source Code from AWS, GitHub, etc...
  • AWS Application Composer - Visually design and build serverless applications quickly on AWS
    • Deploy AWS infrastructure code without needing to be an expert in AWS
    • Configure how your resources interact with each other
    • Generates Infrastructure as Code (IaC) using CloudFormation
  • AWS Device Farm - Fully-managed service that tests your web and mobile apps against desktop browsers, real mobile devices and tablets
    • Run tests concurrently on multiple devices
    • Ability to configure device settings (GPS, language, wi-fi, bluetooth)
  • AWS Backup - Fully managed service to centrally manage and automate backups across AWS services
    • Backup on-demand or at scheduled intervals
    • Supports PITR (Point-in-time Recovery)
    • Configure retention periods, lifecycle management, backup policies...
    • Can backup across regions or accounts (using AWS Organizations)
  • AWS Elastic Disaster Recovery (DRS) - Quickly and easily recover your physical, virtual and cloud-based servers into AWS
  • AWS Fault Injection Simulator (FIS) - Fully managed service for running fault injection experiments on AWS workloads
    • Based on chaos engineering (stressing an application by creating disruptive events, observing how the system responds and implementing improvements)
    • Supports EC2, ECS, EKS, RDS, ...
    • Use pre-built templates that generate the desired disruptions
  • AWS Step Functions - Build a serverless visual workflow to orchestrate your Lambda functions
    • Features: sequence, parallel, conditions, timeouts, error handling
    • Can integrate with EC2, ECS, on-prem servers, API Gateway, SQS queues, etc
    • Use cases - Order fulfillment, data processing, web applications, any workflow
  • AWS Ground Station - Fully managed service that lets you control satellite communications, process data and scale your satellite operations
    • Leverages a global network of satellite ground stations near AWS regions
    • Allows you to download satellite data to your AWS VPC within seconds
    • Send satellite data to S3 or EC2 instance
    • Use cases - Weather forecasting, surface imaging, communications, video broadcasts
  • Amazon Pinpoint - Scalable two-way marketing communications service, supporting email, SMS, push notifications, voice and in-app messaging
    • Provides capabilities to personalize messages with the right content for each customer
    • Can receive replies
    • Scales to billions of messages per day
    • Use cases:
      • Run campaigns by sending marketing, bulk messages, transactional SMS messages
    • Versus SNS/SES:
      • In SNS & SES, you manage each message's audience, content and delivery schedule
      • In Pinpoint, you create message templates, delivery schedules, highly-target segments and full campaigns

Migration

  • AWS DataSync - Move large amount of data from on-prem
    • Can synchronize to Amazon S3, EFS, FSx for Windows
    • Replication tasks can be schedule hourly, daily, weekly
    • Replication tasks are incremental after the first full load
  • AWS Application Discovery Service - Plan migration projects by gathering information about on-prem DCs
    • Important to take into account server utilization data and dependency mapping when migrating.
    • AWS Agentless Discovery Connector - VM inventory, configuration and performance history (eg CPU, memory and disk usage)
    • AWS Agent-based Discovery Agent - System configuration, system performance, running processes and details of the network connections between systems
    • View resulting data in AWS Migration Hub
  • AWS Application Migration Service (MGN) - Service for rehosting solutions, simplifying migrating of applications to AWS
    • Converts your physical, virtual and cloud-based servers to run natively on AWS
    • Supports a wide range of platforms, OSes and DBs
    • Minimal downtime, reduced costs
  • AWS Migration Evaluator - Central location to collect servers and applications inventory data for the assessment, planning and tracking of migrations to AWS.
    • AWS Migration Hub Orchestrator provides pre-built templates to save time and effort migrating enterprise apps
    • Supports migration status updates from Application Migration Service (MGN) and Database Migration Service (DMS)

AWS Architecting & Ecosystem

  • Well-Architected Framework General Principles:
    • Stop guessing your capacity needs
    • Test systems at production scale
    • Automate to make architectural experimentation easier
    • Allow for revolutionary architectures (design based on changing requirements)
    • Drive architectures using data
    • Improve through 'game days' (simulate days with high amounts of system stress, eg flash sale days)
  • Right Sizing is the process of matching instance types and sizes to your workload performance and capability requirements at the lowest possible cost. Scaling up is easy so always start small.
    • Also the process of looking at deployed instances and identifying opportunities to eliminate or downsize without compromizing capacity or other requirements which results in lower costs
    • Important to Right Size before a Cloud Migration and continuously after the cloud onboarding process (as requirements change over time)
    • CloudWatch, Cost Explorer, Trusted Advisor and 3rd party tools can help
  • AWS Cloud Best Practices - Design Principles
    • Scalability (vertical and horizontal)
    • Disposable Resources (servers should be disposable and easily configured)
    • Automation (serverless, IaaS, auto-scaling...)
    • Loose coupling
      • Monoliths are applications that do more and more over time, becoming bigger
      • Break down monolithic applications into smaller, loosely coupled components
      • A change or failure in one component should not cascade to other components
    • Services, not servers (don't just use EC2, use managed services, databases, serverless, etc)

Six Pillars of Well-Architected Frameworks

  • Operational Excellence - includes the ability to run and monitor systems to deliver business value and to continually improve supporting processes and procedures. Design principles:
    • Perform operations as code
    • Annotate documentation - Automate the creation of annotated documentation after each build
    • Make frequent, small, reversible changes in case of failure
    • Refine operations procedures frequently
    • Anticipate failure
    • Learn from all operational failures
  • Security - Includes the ability to protect information, systems and assets while delivering business value through risk assessments and mitigation strategies. Design principles:
    • Implement a strong identity foundation - Centralize privilege management and reduce (or eliminate) reliance on long-term credentials (principle of least privilege)
    • Enable traceability - Integrate logs and metrics with systems to automatically respond and take action
    • Apply security at all layers at edge network, VPC, subnet, load balancer, every instance, OS and application
    • Automate security best practices
    • Protect data in transit and at rest encrypt, tokenize, control access
    • Keep people away from data - Reduce or eliminate the need for direct access or manual processing of data
    • Prepare for security events by running incident response simulations and use tools with automation to increase your speed for detection, investigation and recovery
    • Shared Responsibility Model
  • Reliability - The ability of a system to recover from infrastructure or service disruptions, dynamically acquire computing resources to meet demand and mitigate disruptions such as misconfigurations or transient network issues. Design Principles:
    • Test recovery procedures by using automation to simulate different failures or to recreate scenarios that led to failures before
    • Automatically recover from failure by anticipating and remediating failures before they occur
    • Scale horizontally to increase aggregate system availability - Distribute requests across multiple smaller resources to ensure they don't share a common point of failure
    • Stop guessing capacity - Use auto-scaling
    • Manage change in automation - Use automation to make changes to infrastructure
  • Performance Efficiency - Includes the ability to use computing resources efficiently to meet system requirements and to maintain that efficiency as demand changes and technologies evolve. Design principles:
    • Democratize advanced technologies - Advanced technologies become services and hence you can focus more on product development
    • Go global in minutes - Easy deployment in multiple regions
    • Use serverless architectures to avoid burden of managing servers
    • Experiment more often - Carry out comparative testing
    • Mechanical sympathy - Be aware of all AWS services
  • Cost Optimization - Includes the ability to run systems to deliver business value at the lowest price point. Design principles:
    • Adopt a consumption mode - Pay only for what you use
    • Measure overall efficiency (use CloudWatch)
    • Stop spending money on DC operations - AWS does the infrastructure part and enables customers to focus on organization projects
    • Analyze and attribute expenditure - Accurate identification of system usage and costs helps measure ROI. Make sure to use tags
    • Use managed and application-level services to reduce cost of ownership
  • Sustainability - Focuses on minimizing the environmental impacts of running cloud workloads
    • Understand your impact - Establish performance indicators, evaluate improvements
    • Establish sustainability goals - Set long-term goals for each workload, model ROI
    • Maximize utilization - Right size each workload to maximize energy efficiency of the underlying hardware and minimize idle resources
    • Anticipate and adopt new, more efficient hardware and software offerings and design for flexibility to adopt new technologies over time
    • Use managed services - Shared services reduce the amount of infrastructure, whereas managed services help automate sustainability best practices such as moving infrequent accessed data to cold storage and adjusting compute capacity
    • Reduce the downstream impact of your cloud workloads - Reduce the amount of entergy or resources required to use your services and reduce the need for your customers to upgrade their devices
  • AWS Well-Architected Tool - Free tool to review your architectures against the 6 pillars WAF and adopt architectural best practices

AWS Cloud Adoption Framework

  • AWS Cloud Adoption Framework (CAF) helps you build and execute a comprehensive plan for your digital transformation through innovative use of AWS. Identifies specific organizational capabilities that underpin successful cloud transformations
  • AWS CAF groups its capabilities in six perspectives:
    • The Business Perspective helps ensure that your cloud investments accelerate your digital transformation ambitions and business outcomes
    • The People Perspective is a bridge between tech and business, accelerate the cloud journey to help organizations more rapidly evolve to a culture of continuous growth, learning and where change becomes business-as-normal with a focus on culture, org structure, leadership and workforce.
    • The Governance Perspective helps you orchestrate your cloud initiatives while maximizing organizational benefits and minimizing transformation-related risks
    • The Platform Perspective helps you build an enterprise-grade, scalable hybrid cloud platform, modernize existing workloads and implement new cloud-native solutions
    • The Security Perspective helps you achieve the confidentiality, integrity and availability of your data and cloud workloads
    • The Operations Perspective helps ensure that your cloud services are delivered at a level that meets the needs of your business
  • AWS CAF defines four transformation domains:
    • Technology - using the cloud to migrate and modernize legacy infrastructure, apps, data and analytics platforms
    • Process - digitizing, automating and optimizing your business operations
      • Leveraging new data and analytics platforms to create actionable insights
      • Using ML to improve your customer service experience
    • Organization - Reimagining your operating model
      • Organize your teams around products and value streams
      • Leverage agile methods to rapidly iterate and evolve
    • Product - Reimagining your business model by creating new value propositions (products & services) and revenue models
  • AWS CAF defines four transformation phases
    • Envision how the cloud will accelerate business outcomes by identifying transformation opportunities
    • Align - Identify capability gaps across the 6 AWS CAF perspectives and create an Action Plan
    • Launch pilot initiatives in production and demonstrate incremental business value
    • Scale up pilot initiatives to the desired scale while realizing the desired business benefits

Ecosystem

  • AWS Support
  • AWS Marketplace - Digital catalog with thousands of software listings from independent software vendors
    • If you buy through the AWS Marketplace it goes into your AWS bill
    • You can sell your own solutions on the AWS Marketplace
  • AWS Professional Services is a global organization of experts who can work alongside your team and a chosen member of the AWS Partner Network
    • APN Technology Partners can provide hardware, connectivity and software
    • APN Consulting Partners: Professional services to help build on AWS (like j1)
    • APN Training Partners
    • AWS Competency Program - AWS competencies are granted to APN Partners who have demonstrated technical proficiency and proven customer success in specialized solution areas
    • AWS Navigate Program Partners who partner with partners to help those partners become better partners
  • AWS IQ - Engage and pay AWS Certified 3rd party experts for on-demand project work
    • Service provides video-conferencing, contract management, secure collaboration, integrated billing.
    • For customers:
      • Submit your request
      • Review responses
      • Select expert
      • Work securely
      • Pay per milestone
    • For experts:
      • Create profile
      • Connect with customers
      • Start a proposal
      • Work securely
      • Get paid per milestone
  • AWS re:Post - AWS-managed Q&A service offering crowd-sourced expert-reviewed answers to your technical questions about AWS that replaces the original AWS forums
    • Basically stack overflow
    • Questions from AWS Premium Support customers that do not receive a response from the community are passed on to AWS Support engineers
    • Not intended to be used for questions that are time-sensitive or involve any proprietary information
  • AWS Managed Services - A team of experts offered by AWS who can manage and operate your infrastructure for security, reliability and availability
    • Helps orgs offload routine management tasks and focus on their business objectives
    • Implements best practices and maintains your AWS infrastructure to reduce your operational overhead and risk