Seven Reasons Why RightSizing AWS EBS Capacity Is Hard

Right-sizing EBS volumes can provide a handsome return on investment. The reason is simple: AWS charges for provisioned capacity, not consumed capacity, and almost all AWS customers over-provision their EBS volumes. In our experience, the average amount of unused storage is about 75%. That’s a lot of empty storage that AWS customers are paying for!

By adjusting provisioned capacity to match the actual amount of data stored, FittedCloud has seen customers reduce their EBS spending by up to 70%. It seems straight-forward, but right-sizing EBS volumes can be a complex, labor-intensive proposition. Data sets are rarely static – the amount of data stored by most workloads is constantly changing. Right-sizing requires constant attention to storage utilization in order to ensure applications are neither starved nor gorging on excess capacity. Let’s explore the provisioning issues in detail.

1. Shrinking EBS Volumes Is Not Easy:  Unlike expansion, AWS doesn’t provide a way to reduce the size of a volume. Reducing size of EBS volumes require creating a smaller sized volume and moving the data to the new volume. This can take hours or days depending on the size of EBS volumes.

2. Application Down Time: The procedure for shrinking an EBS volume requires application down time as most filesystems do not support live volume shrinking (even if they did, it doesn’t help as AWS doesn’t support shrinking volumes as mentioned above). This is often a problem for production environments that cannot take downtime. Most customers with such requirements end up stuck with over provisioned EBS volumes.

3. Managing growth: Often EBS volumes are over-provisioned to support future growth. Expanding file systems, and updating applications can be a tedious process and hence customers take the shortcut of over provisioning.  When volumes are right-sized, storage capacity utilization needs to be monitored constantly to expand capacity as needed. (Fortunately, AWS allows in-place expansion of EBS volumes, which makes the process much simpler). This creates an additional overhead for the team that manages the application infrastructure. Errors in provisioning in a timely manner can cause significant disruptions to applications.

4. Root Volume Complexity: Unlike data volumes, root volume capacity is configured by default when a new instance is spun-up. AWS configures 8 GB root volumes for Linux instances and 10GB for Windows instances, even though the OS may require less than 2GB or 3GB. The wasted 6GB or 7GB of EBS space adds up quickly for customers that spin up a large number of instances to run short term workloads, or a large number of instances to run short running workloads.

Many customers also take shortcuts and use large root volumes instead of creating additional data volumes, complicating the matter.

Rightsizing root volumes require a different process than the data volumes as it involves setting up bootable partitions. Once rightsized, customers could also  create custom AMIs that use the appropriate capacity for their OS, so that future instances will use correctly sized root volumes. These blogs describe how this can be done:

How to get around the 8GB limit on root volumes in AWS?
How to reduce the size of EBS Windows boot volume in AWS?

5. Pre-configured AMIs/Applications: Many applications that are distributed as AMI use pre-configured EBS capacity that cannot be reduced below the specified size, even though consumption may be significantly lower than this minimum configuration.

6. LVMs and Partitions Increase Complexity: Even though Logical Volume Managers provide no benefit in public clouds (as they offer virtually unlimited storage capacities), many customers continue to use them with legacy applications and configurations (an LVM configured on top of a volume with a single partition is amusing!). Right-sizing configurations that involve partitions and LVMs can be challenging and adds steps to the process.

7. Scaling-up a Right-Sizing Initiative: An enterprise environment may be composed of a large number of EC2 instances and EBS volumes, making right-sizing a huge task. Each volume needs to be monitored and manually adjusted, which introduces the possibility of human errors. Right-sizing at scale requires tools and automation.

While right-sizing EBS volumes can be difficult, the rewards are significant. FittedCloud has seen customers reduce their EBS bills by up to 70%. The key to building a successful right-sizing initiative is automation. Right-sizing should done done such that it is transparent, doesn’t require application downtime or manual intervention and is continuous. This allows customers to right-size at scale and eliminate the risks of human errors.

FittedCloud’s EBS optimization capabilities include automation and machine learning. It monitors EBS volumes and automatically right-sizes capacity without manual intervention or application disruption. Machine learning ensures applications always have the right amount of storage available. Visit fittedcloud.com for more information.

Other EBS related Blogs