Graduate Program KB

Section 5 - RDS, Aurora & ElastiCache

RDS

  • Relational Database Service is a managed service for databases using SQL as a query language
  • Create databases such as Postgres, MySQL, MariaDB, Oracle, Microsoft SQL Server, IBM DB2 and Aurora (AWS proprietary database)
  • Better to use RDS over deploying a DB on EC2
    • Automated provisioning, OS patching
    • Continuous backups
    • Dashboards
    • Read replicas for improved performance
    • Multi AZ setup for disaster recovery
    • Scaling
    • EBS storage

RDS Auto Scaling

  • Increase RDS DB storage dynamically, you want to avoid manual scaling
  • Set a Maximum Storage Threshold to set a DB storage limit
  • Useful for unpredictable workloads, supports all RDS database engines

RDS Read Replicas

  • Can have up to 15 within AZ, Cross AZ or Cross Region
  • Replication is asynchronous so reads are eventually consistent
  • Replicas can be promoted to their own DB
  • Applications must update the connection string to leverage read replicas
  • There's a network cost when data goes from one AZ to another, for RDS Read Replicas within the same region, there is no fee
  • Use cases:
    • Production database taking on a normal load
    • Run a reporting app to run analytics
    • Want a replica to run new workload
    • Want the production app to be unaffected
    • Read replicas are used for SELECT kind of statements, not INSERT, UPDATE and DELETE

RDS Multi AZ (Disaster Recovery)

  • Synchronous replication, increased availability
  • Failover incase of loss of AZ, network, instance or storage failure
  • No manual intervention in apps, not used for scaling
  • Moving from a Single-AZ to Multi-AZ setup has zero downtime
    • Takes a snapshot and restores the new DB in a new AZ
    • Synchronization is established between the two databases

RDS Custom

  • Access to underlying databases and OS
    • Configure settings
    • Install patches
    • Enable native features
    • Access underlying instance using SSH or SSM Session Manager
  • Better to de-active automation mode to perform customization, also take a DB snapshot prior

Aurora

  • Supports Postgres and MySQL
  • 5 and 3 times more performance over MySQL on RDS and Postgres on RDS respectively
  • Storage automatically grows in increments of 10GB up to 128TB
  • Up to 15 replicas, the replication process is faster than MySQL
  • Failover is instantaneous
  • Cost about 20% more than RDS, but more efficient

Aurora High Availability and Read Scaling

  • 6 copies of data across 3 AZ, 4 are needed for writes and 3 are needed for reads
  • Master Aurora instance takes writes
  • Automated failover for master in less than 30 seconds
  • Support for Cross Region Replication

Aurora Custom Endpoints

  • Define a subset of instances as a custom endpoint
  • Ex. run analytical queries on specific replicas

Aurora Serverless

  • Automated database instantiation and auto-scaling based on actual usage
  • Good for infrequent, intermittent or unpredictable workloads
  • Can be more cost-effective, pay-per-second

Aurora Machine Learning

  • Add ML-based predictions to your apps via SQL
  • Simple and secure integration between Aurora and AWS ML services (SageMaker and Comprehend)
  • Useful for fraud detection, ads targeting, sentiment analysis, product recommendations, etc.

Backups

  • RDS
    • Automated:
      • Daily full backup of DB
      • Transaction logs are backed up every 5 minutes
      • Restore to any point in time
      • 1 to 35 days of retention
    • Manual:
      • Triggered snapshot by user
      • Retention of backup for as long as you want
  • Aurora
    • Automated:
      • 1 to 35 days (can't be disabled)
      • Point-in-time recovery in that time-frame
    • Manual:
      • Triggered snapshot by user
      • Retention of backup for as long as you want

RDS and Aurora Restore Options

  • Restoring a backup or snapshot creates a new DB
  • Restoring MySQL RDS DB from S3
    • Create backup of on-premises DB, store it on S3 and restore the backup file onto a new RDS instance running MySQL
  • Restoring MySQL Aurora cluster from S3
    • Create backup of on-premises DB using Percona XtraBackup, store backup file on S3 and restore it onto a new Aurora cluster running MySQL

Aurora Database Cloning

  • Create new DB Cluster from an existing cluster, faster than snapshot and restore
  • Uses copy-on-write protocol
  • Fast and cost-effective
  • Useful for creating a staging database from a production database without impacting the production database

RDS and Aurora Security

  • At-rest encryption: Use AWS KMS defines at launch time
  • In-flight encryption: TLS-ready by default, use TLS Root certificates client-side
  • IAM authentication: IAM roles to connect to DB
  • Security groups: Control network access to database
  • No SSH available except on RDS custom
  • Audit logs can be enabled and sent to CloudWatch Logs for longer retention

RDS Proxy

  • Enables apps to pool and share established DB connections
  • Serverless, auto scaling, high availability with Multi-AZ setup
  • Improves DB efficiency by reducing stress on DB resources and minimizing open connections
  • Reduced RDS and Aurora failover time by up to 66%
  • Proxy only accessible from VPC, not public

ElastiCache

  • In-memory database with high performance and low latency
  • AWS managed service used to get managed Redis or Memcached
  • Reduce read intensive workloads, make apps stateless
  • Using ElastiCache involves heavy application code changes
    • Apps query ElastiCache, if it's not available then query RDS and store in ElastiCache
    • Cache must invalidate stale data

User Session Store with ElastiCache

  • User logs into app
  • App writes session data into ElastiCache
  • User hits another instance of the app
  • Instance retrieves the data and the user is already logged in

ElastiCache Redis vs. Memcached

  • Redis
    • Multi-AZ with auto-failover
    • Read Replicas to scale reads and high availability
    • Backup and restore features
    • Support sets and sorted sets
  • Memcached
    • Sharding: Multi-node for partitioning of data
    • No replication
    • Not persistent
    • Serverless backup and restore
    • Multi-threaded architecture

ElastiCache Cache Security

  • Supports IAM authentication for Redis
  • IAM policies only used for AWS API-level security
  • Redis AUTH:
    • Set a password/token upon creating a cluster
    • Support SSL in-flight encryption
  • Memcached:
    • Supports SASL-based authentication

ElastiCache Patterns

  • Lazy loading: Cache all read data, but data can become stale
  • Write through: Adds or update data in cache when written to DB (solves stale data)
  • Session store: Store temporary session data using TTL features