Use Instance Store With AWS Elastic Container Storage

Many EC2 instance types come with instance attached storage (Instance Store) which can provide a fast local storage that is faster than using an EBS volume. If your using the Amazon ECS-optimized AMI (Amazon Linux 1), it’s instance storage is a secondary EBS volume that is used for storing docker containers and volumes. If your launching it on an EC2 with instance store, it is ignored and only the one EBS volume is used. So here is a simplistic cloud-init script that detects the attached SSD and NVMe SSD’s and adds them to the LVM volume group.
Note: I have not done thorough performance testing but have noticed a slight increase in IO times when configured this way on the newer generation (i.e. c5d, r5d etc) types.

Just launch your EC2 instance with the following user data or download the script from this gist if you’ve got a more complex init script already.

EC2 User Data
#cloud-boothook
set -ex


# find ephemeral devices using the metadata service
# note: this will block loading until the metadata service is ready
find_ephemeral_devices() {
    echo "Querying metadata instance store for ephemeral volumes" <&2
    local DEVICE_NOT_FOUND=0
    for d in $(curl -s "http://169.254.169.254/latest/meta-data/block-device-mapping/" | grep ephemeral); do
       NAME=$(curl -s "http://169.254.169.254/latest/meta-data/block-device-mapping/$d")
       DEVICE_NAME="/dev/$NAME"
       echo "Detected ephemeral device $d corresponding to $DEVICE_NAME" <&2
       # if block device (-b)
       if [[ -b "$DEVICE_NAME" ]]; then
          echo "$DEVICE_NAME"
          DEVICE_NOT_FOUND=0
       else
          echo "Couldn't find device $DEVICE_NAME" <&2
          DEVICE_NOT_FOUND=1
       fi
    done
    return ${DEVICE_NOT_FOUND}
}

# on nvme instances the device names don't match what AWS reports, so just filter by model name
find_nvme_ephemeral() {
  lsblk -o KNAME,MODEL | grep "Amazon EC2 NVMe Instance Storage" | awk '{ print "/dev/"$1 }'
}

METADATA_DEVICES=$(find_ephemeral_devices)
NVME_DEVICES=$(find_nvme_ephemeral)

DEVS=/dev/xvdcz
[[ ! -z "$NVME_DEVICES" ]] && DEVS="$NVME_DEVICES $DEVS"
[[ ! -z "$METADATA_DEVICES" ]] && DEVS="$METADATA_DEVICES $DEVS"

echo "Updating docker storage devices to '$DEVS'"
sed -i -e "s;DEVS=.*;DEVS=\"${DEVS}\";g"  /etc/sysconfig/docker-storage-setup

Comments