Graduate Program KB

Section 3 - EC2 Instance Storage

Same content! Refer to Section 04 - EC2 Instance Storage

EBS Volume Types

  • Characterised in size, throughput, IOPS (I/O operations per second)
  • There are 6 types:
    • gp2 / gp3 (SSD): General purpose SSD volume balancing price and performance for a variety of tasks (can be used as boot volume)
    • io1 / io2 Block Express (SSD): Best performing SSD volume for mission-critical, low-latency or high-throughput workloads (Can be used as boot volume)
    • st1 (HDD): Cheap HDD volume for frequently accessed, throughput-intensive workloads
    • sc1 (HDD): Lowest cost HDD volume for less frequently accessed workloads
  • Use cases:
    • General purpose SSD
      • Cost effective storage and low-latency
      • System boot volumes, virtual desktops, development and test environments
    • Provisioned IOPS (PIOPS) SSD
      • Critical business applications with sustained IOPS performance
      • Applications that need more than 16,000 IOPS
      • Database workloads (require storage performance and consistency)
      • Supports EBS multi-attach
    • HDD
      • Can't be a boot volume

EBS Multi-Attach

  • Attach the same EBS volume to multiple EC2 instances in the same AZ
  • Each instance has full read / write permissions to the volume
  • Use case:
    • Achieve higher application availability in clustered Linux applications
    • Applications msut manage concurrent write operations
  • Can attach up to 16 instances at a time
  • Need to use a file system that's cluster-aware (not XFS, EXT4, etc...)

EBS Encryption

  • Upon creation of an encrypted EBS volume
    • Data at rest is encrypted inside the volume
    • All data in flight moving between the instance and volume is encrypted
    • All snapshots are encrypted
    • All volumes created from the snapshot
  • Encryption and decryption are handled by AWS
  • Encryption has minimal impact on latency
  • Leverages keys from KMS (AES-256)
  • Copying an unencrypted snapshot allows encryption
  • Snapshots of encrypted volumes are encrypted
  • To encrypt an unencrypted EBS volume:
    • Create an EBS snapshot of the volume
    • Encrypt the EBS snapshot using the copy
    • Create a new EBS volume from the snapshot (the volume will also be encrypted)
    • Can attach the encrypted volume to the original instance now

Elastic File System

  • Managed NFS that's mountable to many instances
  • Works with instances in multiple AZ
  • Highly available, scalable (automatic), expensive, pay per use
  • Uses NFSv4.1 protocol and a security group to control access to EFS
  • Compatible with Linux based AMI (not Windows)
  • Encryption at rest using KMS
  • Use cases:
    • Content managemenet, web serving, data sharing, WordPress

EFS Performance & Storage Classes

  • EFS Scale
    • Thousands of concurrent NFS clients, 10 GB+ /s throughput
    • Can grow to a Petabyte-scale NFS automatically
  • Performance Mode
    • Set at EFS creation time
    • General purpose (default): Latency-sensitive use cases (web server, CMS)
    • Max I/O: Higher latency, throughput, parallel (big data, media processing)
  • Throughput Mode
    • Bursting: Can suddenly burst up to 100 MiB/s from 50 MiB/s
    • Provisioned: Set a fixed throughput regardless of storage size
    • Elastic: Automatically scales throughput up or down, great for unpredictible workloads
  • Storage Tiers (lifecycle management)
    • Standard: Frequently accessed files
    • Infrequent access (EFS-IA): Cost to retrieve files, lower price to store
    • Archive: Rarely accessed data (few times per year), 50% cheaper
    • Leverage lifecycle policies to move files between tiers
  • Availability and durability
    • Standard: Multi-AZ, great for production
    • One Zone: Single AZ, great for development, backup enabled by default and compatible with IA (EFS One Zone-IA)
  • Over 90% in cost savings

EBS vs EFS

  • EBS
    • Used on one instance (except multi-attach io1 / io2)
    • Locked at the AZ level
    • Migrating an EBS volume across an AZ requires a snapshot and restoring the snapshot in another AZ
    • Root EBS volumes of instances get terminated by default if the instance gets terminated (can be disabled)
  • EFS
    • Can mount to hundreds of instances across different AZ
    • EFS share website files (WordPress)
    • Only for Linux instances (POSIX)
    • Higher price point than EBS
    • Leverage Storage Tiers for cost savings