Section 5 - RDS, Aurora & ElastiCache
RDS
- Relational Database Service is a managed service for databases using SQL as a query language
- Create databases such as Postgres, MySQL, MariaDB, Oracle, Microsoft SQL Server, IBM DB2 and Aurora (AWS proprietary database)
- Better to use RDS over deploying a DB on EC2
- Automated provisioning, OS patching
- Continuous backups
- Dashboards
- Read replicas for improved performance
- Multi AZ setup for disaster recovery
- Scaling
- EBS storage
RDS Auto Scaling
- Increase RDS DB storage dynamically, you want to avoid manual scaling
- Set a Maximum Storage Threshold to set a DB storage limit
- Useful for unpredictable workloads, supports all RDS database engines
RDS Read Replicas
- Can have up to 15 within AZ, Cross AZ or Cross Region
- Replication is asynchronous so reads are eventually consistent
- Replicas can be promoted to their own DB
- Applications must update the connection string to leverage read replicas
- There's a network cost when data goes from one AZ to another, for RDS Read Replicas within the same region, there is no fee
- Use cases:
- Production database taking on a normal load
- Run a reporting app to run analytics
- Want a replica to run new workload
- Want the production app to be unaffected
- Read replicas are used for SELECT kind of statements, not INSERT, UPDATE and DELETE
RDS Multi AZ (Disaster Recovery)
- Synchronous replication, increased availability
- Failover incase of loss of AZ, network, instance or storage failure
- No manual intervention in apps, not used for scaling
- Moving from a Single-AZ to Multi-AZ setup has zero downtime
- Takes a snapshot and restores the new DB in a new AZ
- Synchronization is established between the two databases
RDS Custom
- Access to underlying databases and OS
- Configure settings
- Install patches
- Enable native features
- Access underlying instance using SSH or SSM Session Manager
- Better to de-active automation mode to perform customization, also take a DB snapshot prior
Aurora
- Supports Postgres and MySQL
- 5 and 3 times more performance over MySQL on RDS and Postgres on RDS respectively
- Storage automatically grows in increments of 10GB up to 128TB
- Up to 15 replicas, the replication process is faster than MySQL
- Failover is instantaneous
- Cost about 20% more than RDS, but more efficient
Aurora High Availability and Read Scaling
- 6 copies of data across 3 AZ, 4 are needed for writes and 3 are needed for reads
- Master Aurora instance takes writes
- Automated failover for master in less than 30 seconds
- Support for Cross Region Replication
Aurora Custom Endpoints
- Define a subset of instances as a custom endpoint
- Ex. run analytical queries on specific replicas
Aurora Serverless
- Automated database instantiation and auto-scaling based on actual usage
- Good for infrequent, intermittent or unpredictable workloads
- Can be more cost-effective, pay-per-second
Aurora Machine Learning
- Add ML-based predictions to your apps via SQL
- Simple and secure integration between Aurora and AWS ML services (SageMaker and Comprehend)
- Useful for fraud detection, ads targeting, sentiment analysis, product recommendations, etc.
Backups
- RDS
- Automated:
- Daily full backup of DB
- Transaction logs are backed up every 5 minutes
- Restore to any point in time
- 1 to 35 days of retention
- Manual:
- Triggered snapshot by user
- Retention of backup for as long as you want
- Aurora
- Automated:
- 1 to 35 days (can't be disabled)
- Point-in-time recovery in that time-frame
- Manual:
- Triggered snapshot by user
- Retention of backup for as long as you want
RDS and Aurora Restore Options
- Restoring a backup or snapshot creates a new DB
- Restoring MySQL RDS DB from S3
- Create backup of on-premises DB, store it on S3 and restore the backup file onto a new RDS instance running MySQL
- Restoring MySQL Aurora cluster from S3
- Create backup of on-premises DB using Percona XtraBackup, store backup file on S3 and restore it onto a new Aurora cluster running MySQL
Aurora Database Cloning
- Create new DB Cluster from an existing cluster, faster than snapshot and restore
- Uses copy-on-write protocol
- Fast and cost-effective
- Useful for creating a staging database from a production database without impacting the production database
RDS and Aurora Security
- At-rest encryption: Use AWS KMS defines at launch time
- In-flight encryption: TLS-ready by default, use TLS Root certificates client-side
- IAM authentication: IAM roles to connect to DB
- Security groups: Control network access to database
- No SSH available except on RDS custom
- Audit logs can be enabled and sent to CloudWatch Logs for longer retention
RDS Proxy
- Enables apps to pool and share established DB connections
- Serverless, auto scaling, high availability with Multi-AZ setup
- Improves DB efficiency by reducing stress on DB resources and minimizing open connections
- Reduced RDS and Aurora failover time by up to 66%
- Proxy only accessible from VPC, not public
ElastiCache
- In-memory database with high performance and low latency
- AWS managed service used to get managed Redis or Memcached
- Reduce read intensive workloads, make apps stateless
- Using ElastiCache involves heavy application code changes
- Apps query ElastiCache, if it's not available then query RDS and store in ElastiCache
- Cache must invalidate stale data
User Session Store with ElastiCache
- User logs into app
- App writes session data into ElastiCache
- User hits another instance of the app
- Instance retrieves the data and the user is already logged in
ElastiCache Redis vs. Memcached
- Redis
- Multi-AZ with auto-failover
- Read Replicas to scale reads and high availability
- Backup and restore features
- Support sets and sorted sets
- Memcached
- Sharding: Multi-node for partitioning of data
- No replication
- Not persistent
- Serverless backup and restore
- Multi-threaded architecture
ElastiCache Cache Security
- Supports IAM authentication for Redis
- IAM policies only used for AWS API-level security
- Redis AUTH:
- Set a password/token upon creating a cluster
- Support SSL in-flight encryption
- Memcached:
- Supports SASL-based authentication
ElastiCache Patterns
- Lazy loading: Cache all read data, but data can become stale
- Write through: Adds or update data in cache when written to DB (solves stale data)
- Session store: Store temporary session data using TTL features