JavaScript is turned off in your web browser. To take full advantage of Ribose features, please enable JavaScript and refresh the page. Ribose > Tech post: AWS: Converting CentOS and Red Hat AMIs to EXT4

Tech post: AWS: Converting CentOS and Red Hat AMIs to EXT4

It’s a well known fact that Red Hat and CentOS use XFS for their default file system so it should come as no surprise that this default is reflected in the most frequently used Red Hat-based AMIs on AWS EC2. These distributions used to default to EXT4 but that time has long past.

However, there are several reasons you may want to use EXT4 as your default file system, and the first of that on AWS EC2 is because of performance. Yet it is something you must explicitly define at time of install or convert the system after the install has finished. You do not get to make this choice with the AMIs on AWS and that can be a real bummer if you are looking to use a file system other than XFS.

It’s a great time to be alive though. Why? Because we have the tools to make this happen ourselves. The tool we are specifically interested in here is Packer. Packer can take an existing AMI and build a new AMI based on settings we define as well as apply provisioner scripts to alter the underlying image. Provisioner scripts don’t have to be complicated and, in this case, we can write a bash script that will convert a root CentOS or Red Hat partition from XFS to EXT4.

To talk about this process from a Packer standpoint, we will tell Packer to do several things in order:

  1. Spin up an instance from a Red Hat or CentOS AMI of our choosing and attach an extra storage volume

  2. Run the provisioner script that will also do several things:

    1. format the extra volume with EXT4

    2. mount the volume

    3. rsync all relevant files from the XFS root volume to the EXT4 volume

    4. update the EXT4 volume’s fstab to reflect the new boot partition

    5. update the EXT4 volume’s boot directory and Grub configuration

  3. Build a new AMI using the prepared EXT4 volume (which will now be the root volume for instances based of this AMI)

The Packer builder configuration should look something like this for example:

  "variables": {
    "version": "7.3",
    "aws_access_key_id": "{{env `AWS_ACCESS_KEY_ID`}}",
    "aws_secret_access_key": "{{env `AWS_SECRET_ACCESS_KEY`}}",
    "aws_security_token": "{{env `AWS_SECURITY_TOKEN`}}",
    "packer_build_vpc_id": "{{env `PACKER_BUILD_VPC_ID`}}",
    "packer_build_subnet_id": "{{env `PACKER_BUILD_SUBNET_ID`}}"

  "min_packer_version": "0.12.3",

  "builders": [
      "type": "amazon-ebssurrogate",
      "ami_virtualization_type": "hvm",
      "region": "us-east-1",
      "source_ami": "ami-b63769a1",
      "ssh_username": "ec2-user",
      "ssh_pty": "true",
      "access_key": "{{user `aws_access_key_id`}}",
      "secret_key": "{{user `aws_secret_access_key`}}",
      "token": "{{user `aws_security_token`}}",
      "vpc_id": "{{user `packer_build_vpc_id`}}",
      "subnet_id": "{{user `packer_build_subnet_id`}}",
      "ami_name": "redhat-encrypted {{user `version`}} {{timestamp}}",
      "instance_type": "t2.micro",
      "encrypt_boot": "true",

      "launch_block_device_mappings": [
          "device_name": "/dev/sdf",
          "delete_on_termination": "true",
          "volume_type": "gp2",
          "volume_size": "100"

      "ami_root_device": {
        "source_device_name": "/dev/sdf",
        "device_name": "/dev/sda1",
        "delete_on_termination": true,
        "volume_size": "100",
        "volume_type": "gp2"

  "provisioners": [
      "type": "shell",
      "execute_command": "echo 'packer' | {{ .Vars }} sudo -E -S sh '{{ .Path }}'",
      "script": "scripts/"

As you can see, we are mounting an additional volume (/dev/sdf), running our provisioner script that creates the EXT4 root volume, then mounts that volume as if were the original volume (as /dev/sda1) when it comes time to build the new AMI. You’ll also see that we are using the amazon-ebssurrogate builder type that Packer provides. This type exists precisely for the purpose of what we are trying to do here. That’s why you don’t see us attaching the source /dev/sda1 volume. It is assumed that we will be attaching a volume to use as a root volume and provisioning it as we deem necessary based on the source root volume. Lovely.

We’ve walked through the Packer configuration and now you’re probably wondering how the actual XFS to EXT4 conversion happens in the provisioner script. Here’s how we do it in our bash script:

readonly __progname="$(basename $0)"

errx() {
  echo -e "$__progname: $@" >&2
  exit 1

main() {
  readonly local dev="/dev/xvdf"
  readonly local mntpoint="/mnt"

  [ ! -d "${mntpoint}" ] &&
    errx "cannot find mountpoint '${mntpoint}'"

  parted -a optimal "${dev}" mklabel gpt mkpart primary '0%' '1%' name 1 grub set 1 bios_grub on print ||
    exit 1
  parted -a optimal "${dev}" mkpart primary '1%' '100%' name 2 rootfs set 2 boot on print ||
    exit 1
  mkfs.ext3 "${dev}1" ||
    exit 1
  mkfs.ext4 "${dev}2" ||
    exit 1

  mount "${dev}2" "${mntpoint}"

  # amazon-ebssurrogate: rsync: getcwd(): No such file or directory (2)
  # amazon-ebssurrogate: rsync error: errors selecting input/output files, dirs (code 3) at util.c(1008) [Receiver=3.0.9]

  cd /
  rsync -aAXv --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} / "${mntpoint}" ||
    exit 1

  readonly local uuid_d1="$(lsblk -no UUID ${dev}1)"
  readonly local uuid_d2="$(lsblk -no UUID ${dev}2)"

  echo "UUID=${uuid_d2} / ext4 defaults 0 0" > "${mntpoint}/etc/fstab"

  grub2-install --boot-directory="${mntpoint}/boot" "${dev}" ||
    exit 1

  grub2-mkconfig -o "${mntpoint}/boot/grub/grub.cfg" ||
    exit 1

  if [ $(cat /etc/redhat-release | awk '{ print $1 }') == 'CentOS' ]; then
    readonly local oldid="${uuid_d1}"
    readonly local oldid="${uuid_d2}"

  find "${mntpoint}/boot/" -type f -exec sed -i "s/${oldid}/${uuid_d2}/g" {};

  umount "${mntpoint}" ||
    exit 1
  echo "finished!"

main "$@"
exit 0

If you are familiar with bash, then this should make some sense to you based on what talked about above. The additional volume needs to be partitioned and formated with EXT4. We then mount it and rsync the operating system data from the source volume to the new EXT4 volume. We store the UUIDs of the volumes involved in variables and make the appropriate changes to the /etc/fstab, boot files, and GRUB configuration. It is at this point where we can unmount the EXT4 volume and proceed with building the new AMI.