Skip to content

Latest commit

 

History

History
162 lines (125 loc) · 7.25 KB

ec2_storage.md

File metadata and controls

162 lines (125 loc) · 7.25 KB

EC2 Instance Storage

EBS Volumes

What’s an EBS Volume?

  • An EBS (Elastic Block Store) Volume is a network drive you can attach to your instances while they run
  • It allows your instances to persist data, even after their termination
  • They can only be mounted to one instance at a time (at the CCP level)
  • They are bound to a specific availability zone
  • Analogy: Think of them as a “network USB stick”
  • Free tier: 30 GB of free EBS storage of type General Purpose (SSD) or Magnetic per month

EBS Volume

  • It’s a network drive (i.e. not a physical drive)
    • It uses the network to communicate the instance, which means there might be a bit of latency
    • It can be detached from an EC2 instance and attached to another one quickly
  • It’s locked to an Availability Zone (AZ)
    • An EBS Volume in us-east-1a cannot be attached to us-east-1b
    • To move a volume across, you first need to snapshot it
  • Have a provisioned capacity (size in GBs, and IOPS)
    • You get billed for all the provisioned capacity
    • You can increase the capacity of the drive over time

Elastic File System

EBS – Delete on Termination attribute

  • Controls the EBS behaviour when an EC2 instance terminates
    • By default, the root EBS volume is deleted (attribute enabled)
    • By default, any other attached EBS volume is not deleted (attribute disabled)
  • This can be controlled by the AWS console / AWS CLI
  • Use case: preserve root volume when instance is terminated

EBS Snapshots

  • Make a backup (snapshot) of your EBS volume at a point in time
  • Not necessary to detach volume to do snapshot, but recommended
  • Can copy snapshots across AZ or Region

EBS Snapshots Features

  • EBS Snapshot Archive
    • Move a Snapshot to an ”archive tier” that is 75% cheaper
    • Takes within 24 to 72 hours for restoring the archive
  • Recycle Bin for EBS Snapshots
    • Setup rules to retain deleted snapshots so you can recover them after an accidental deletion
    • Specify retention (from 1 day to 1 year)

EFS: Elastic File System

  • Managed NFS (network file system) that can be mounted on 100s of EC2
  • EFS works with Linux EC2 instances in multi-AZ
  • Highly available, scalable, expensive (3x gp2), pay per use, no capacity planning

Elastic File System

EFS Infrequent Access (EFS-IA)

  • Storage class that is cost-optimized for files not accessed every day
  • Up to 92% lower cost compared to EFS Standard
  • EFS will automatically move your files to EFS-IA based on the last time they were accessed
  • Enable EFS-IA with a Lifecycle Policy
  • Example: move files that are not accessed for 60 days to EFS-IA
  • Transparent to the applications accessing EFS

Amazon FSx – Overview

  • Launch 3rd party high-performance file systems on AWS
  • Fully managed service
    • FSx for Lustre
    • FSx for Windows File Server
    • FSx for NetApp ONTAP

Amazon FSx for Windows File Server

  • A fully managed, highly reliable, and scalable Windows native shared file system
  • Built on Windows File Server
  • Supports SMB protocol & Windows NTFS
  • Integrated with Microsoft Active Directory
  • Can be accessed from AWS or your on-premise infrastructure

Amazon FSx for Lustre

  • A fully managed, high-performance, scalable file storage for High Performance Computing (HPC)
  • The name Lustre is derived from “Linux” and “cluster”
  • Machine Learning, Analytics, Video Processing, Financial Modeling
  • Scales up to 100s GB/s, millions of IOPS, sub-ms latencies

EC2 Instance Store

  • EBS volumes are network drives with good but “limited” performance
  • If you need a high-performance hardware disk, use EC2 Instance Store
  • Better I/O performance
  • EC2 Instance Store lose their storage if they’re stopped (ephemeral)
  • Good for buffer / cache / scratch data / temporary content
  • Risk of data loss if hardware fails
  • Backups and Replication are your responsibility

Shared Responsibility Model for EC2 Storage

AWS USER
Infrastructure Setting up backup / snapshot procedures
Replication for data for EBS volumes & EFS drives Setting up data encryption
Replacing faulty hardware Responsibility of any data on the drives
Ensuring their employees cannot access your data Understanding the risk of using EC2 Instance Store

AMI Overview

  • AMI = Amazon Machine Image
  • AMI are a customization of an EC2 instance
    • You add your own software, configuration, operating system, monitoring…
    • Faster boot / configuration time because all your software is pre-packaged
  • AMI are built for a specific region (and can be copied across regions)
  • You can launch EC2 instances from:
    • A Public AMI: AWS provided
    • Your own AMI: you make and maintain them yourself
    • An AWS Marketplace AMI: an AMI someone else made (and potentially sells)

AMI Process (from an EC2 instance)

  • Start an EC2 instance and customize it
  • Stop the instance (for data integrity)
  • Build an AMI – this will also create EBS snapshots
  • Launch instances from other AMIs

EC2 Image Builder

  • Used to automate the creation of Virtual Machines or container images
  • => Automate the creation, maintain, validate and test EC2 AMIs
  • Can be run on a schedule (weekly, whenever packages are updated, etc…)
  • Free service (only pay for the underlying resources)

EC2: Virtual Machines            List           Elastic Load Balancing & Auto Scaling Groups