A Simple Open Source Tool for EC2 RightSizing Analysis

AWS provides a range of monitoring and reporting tools for analyzing resource utilization data at an individual EC2 instance level. However, it is difficult for customers to digest a large amount of information in order to reach meaningful conclusions in an aggregated manner. Many AWS customers are looking for a simple reporting tool which can provide quicker and easier analysis that can help them understand the utilization efficiency of their environment that can lead to immediate rightsizing actions. Such tools are useful to identify potential cost savings and to reduce AWS spend significantly.

FittedCloud just released an Open Source  EC2 Rightsizing Analysis Tool which enables customers to have insights into their EC2 resources and identify overprovisioned EC2 instances. We designed it in such a way that users can run the tools in their environments and don’t need to give out access to third parties. The tool is open sourced on Github.

Use the following command to run the tool:

python ec2stats.py -k YOUR_ACCESS_KEY -s YOUR_SECRET_ACCESS_KEY

It Generates the following output:

The tool reports the following:

  1. Average CPU utilization distribution and Summary:  Distribution shows how your EC2 instances’ average CPU utilization (based on data from 14 days) is distributed over 10% increments.  Summary shows min, max and mean of average CPU utilization, number of instances under <=5% utilization, <=10% utilization and <=30% utilization.
  2. Maximum CPU utilization distribution and Summary: Distribution shows how your EC2 instances’ maximum CPU utilization (based on data from 14 days) is distributed over 10% increments.  Summary shows min, max and mean of maximum CPU utilization, number of instances under <=5% utilization, <=10% utilization and <=30% utilization
  3. Distribution of Instance types: This data shows how your EC2 instances are distributed across various families of EC2 instances in your environment.
  4. Efficiency Score: It gives the efficiency score and compares it to other users with similar EC2 spending.
  5. Instances under the specified average or maximum CPU utilization: This report provides a list of EC2 instances and details of EC2 instances with average or maximum CPU utilization under the specified limit

The REST interface allows customers to integrate the service into their existing AWS management systems. For better visualization of results, an optional web interface is also provided. The web page allows user to upload Cloudwatch stats generated by the utility and get a visualized view of the analysis. It shows the distribution of CPU usage in terms of Average CPU Utilization and Maximum CPU Utilization. The report singles out the numbers of instances with CPU utilization below 5%, 10%, and 30%. It gives customer a clear view of the efficiency of their EC2 instances.

How does one use this data to rightsize EC2 instances?

While there are metrics such as memory utilization, disk utilization, network bandwidth utilization that should be considered for more accurate rightsizing decisions, CPU utilization is a key metric that can be used to determine if an instance is over provisioned. Instances with low maximum CPU utilizations (for e.g. under 10%) are candidates for rightsizing.

Customers may also review disk utilization and network utilization data provided by AWS to come up with more accurate rightsizing decisions. Memory utilization is not currently available from CloudWatch unless custom CloudWatch metric data collection through CloudWatch agent is enabled.

Once instances that are over-provisioned are identified, switch the instance type down by one level first. For e.g. if you are using m4.4xlarge instance type which has been identified as a candidate, you can switch it to m4.2xlarge (and save 50% of the cost).

Other Related Blogs

FittedCloud Cloud Cost Optimization Solutions

FittedCloud offers machine learning based cloud cost optimization solutions. Our current AWS cost optimization solution includes machine learning driven actionable advisories with click through actions for EC2, EBS, RDS, DynamoDB, ElastiCache, ElasticSearch, AutoScale, Lambda, etc. and full/lights out automation for EC2, EBS, DynamoDB and RDS. Data shows that our solutions save customers up to 50% of their cost on AWS. For more details, please visit https://www.fittedcloud.com/.