In a previous blog I demonstrated the steps to reduce the root EBS volume size of a Amazon Linux EC2 instance. Recently I have been asked to do it on a Windows Server instance. With minor tweaks to the steps from my previous blog I was able to reduce the Windows Server’s root volume. With this, not only can we save the EBS usage costs, we also save the backup snapshot costs of such instances. Furthermore, AMIs can be created with minimum spaces. When teams at work routinely launch EC2 instances from such reduced sized AMI images the savings could be significant.
In this blog, I will demonstrate the steps to reduce a Microsoft Windows Server 2016 Base (ami-b2e756ca) root EBS volume size from 30GB to 20GB. The actual file system used size is about 19.7GB.
- Stop the target Windows Server EC2 instance.
- Snapshot the Windows Server’s root EBS volume.
- Create an EBS volume from the snapshot. This will be the 30GB temporary source EBS volume.
- Launch a new EC2 worker instance in the same availability zone as the target instance. Here I launched the instance from Amazon Linux AMI 2017.09.1 HVM (ami-f2d3638a.) Install the ntfsprogs RPM.
yum --enablerepo epel install ntfsprogs
- Attach the source EBS volume to the worker instance as xvdf.
- Dump current source disk partition info to a file. Note the root partition xvdf1 starts at sector 2048.
sfdisk -d /dev/xvdf > sfdisk-d.xvdf # partition table of /dev/xvdf unit: sectors /dev/xvdf1 : start= 2048, size= 62910464, Id= 7, bootable /dev/xvdf2 : start= 0, size= 0, Id= 0 /dev/xvdf3 : start= 0, size= 0, Id= 0 /dev/xvdf4 : start= 0, size= 0, Id= 0
- On the worker instance, find out the minimum size the filesystem can be reduced to.
ntfsresize --info /dev/xvdf1 ntfsresize v2017.3.23 (libntfs-3g) Device name : /dev/xvdf1 NTFS volume version: 3.1 Cluster size : 4096 bytes Current volume size: 32210153984 bytes (32211 MB) Current device size: 32210157568 bytes (32211 MB) Checking filesystem consistency ... 100.00 percent completed Accounting clusters ... Space in use : 19227 MB (61.2%) Collecting resizing constraints ... You might resize at 19721502720 bytes or 19227 MB (freeing 12489 MB). Please make a test run using both the -n and -s options before real resizing!
- Create a new target EBS volume of a smaller size. The size needs to be at least slightly larger than the actual filesystem used size. Here I used 20GB.
- Attach the new target EBS volume to the worker instance as xvdg.
- Resize the filesystem to the suggested 19227MB. Enter ‘y’ when prompted to proceed.
ntfsresize -s 19227M /dev/xvdf1 ntfsresize v2017.3.23 (libntfs-3g) Device name : /dev/xvdf1 NTFS volume version: 3.1 Cluster size : 4096 bytes Current volume size: 32210153984 bytes (32211 MB) Current device size: 32210157568 bytes (32211 MB) New volume size : 19226997248 bytes (19227 MB) Checking filesystem consistency ... 100.00 percent completedd Accounting clusters ... Space in use : 12984 MB (40.3%) Collecting resizing constraints ... Needed relocations : 0 (0 MB) WARNING: Every sanity check passed and only the dangerous operations left. Make sure that important data has been backed up! Power outage or computer crash may result major data loss! Are you sure you want to proceed (y/[n])?y Schedule chkdsk for NTFS consistency check at Windows boot time ... Resetting $LogFile ... (this might take a while) Updating $BadClust file ... Updating $Bitmap file ... Updating Boot record ... Syncing device ... Successfully resized NTFS on device '/dev/xvdf1'. You can go on to shrink the device for example with Linux fdisk. IMPORTANT: When recreating the partition, make sure that you 1) create it at the same disk sector (use sector as the unit!) 2) create it with the same partition type (usually 7, HPFS/NTFS) 3) do not make it smaller than the new NTFS filesystem size 4) set the bootable flag for the partition if it existed before Otherwise you won't be able to access NTFS or can't boot from the disk! If you make a mistake and don't have a partition table backup then you can recover the partition table by TestDisk or Parted's rescue mode.
- Find out the total MB we need to copy from the source disk to the new smaller disk. Here we need to cover the reserved sectors (2048 sectors, or 1MB, from step 6 above) at the front of the disk, to the end of the filesystem.
echo $((1+19227)) 19228
- Block copy from xvdf to xvdg, with 1MB block size and total count of 19228.
dd bs=1M if=/dev/xvdf of=/dev/xvdg count=19228 20162019328 bytes (20 GB) copied, 442.911 s, 45.5 MB/s
- Dump the new target disk xvdg partition info to a file. Edit the file by updating the xvdg1 size to the new size in sectors (39376896.)
echo $((19227*2048)) 39376896 sfdisk -d /dev/xvdg > sfdisk-d.xvdg # partition table of /dev/xvdg unit: sectors /dev/xvdg1 : start= 2048, size=
6291046439376896, Id= 7, bootable /dev/xvdg2 : start= 0, size= 0, Id= 0 /dev/xvdg3 : start= 0, size= 0, Id= 0 /dev/xvdg4 : start= 0, size= 0, Id= 0
- Repartition target disk xvdg.
sfdisk /dev/xvdg < sfdisk-d.xvdg Checking that no-one is using this disk right now ... OK Disk /dev/xvdg: 2610 cylinders, 255 heads, 63 sectors/track Old situation: Units: cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0 Device Boot Start End #cyls #blocks Id System /dev/xvdg1 * 0+ 3916- 3916- 31455232 7 HPFS/NTFS/exFAT /dev/xvdg2 0 - 0 0 0 Empty /dev/xvdg3 0 - 0 0 0 Empty /dev/xvdg4 0 - 0 0 0 Empty New situation: Units: sectors of 512 bytes, counting from 0 Device Boot Start End #sectors Id System /dev/xvdg1 * 2048 39378943 39376896 7 HPFS/NTFS/exFAT /dev/xvdg2 0 - 0 0 Empty /dev/xvdg3 0 - 0 0 Empty /dev/xvdg4 0 - 0 0 Empty Warning: partition 1 does not end at a cylinder boundary Successfully wrote the new partition table Re-reading the partition table ... If you created or changed a DOS partition, /dev/foo7, say, then use dd(1) to zero the first 512 bytes: dd if=/dev/zero of=/dev/foo7 bs=512 count=1 (See fdisk(8).)
- Detach xvdg from the worker instance, attach it to the target instance as /dev/sda1, and create an AMI image from the instance.
Verify everything is good before proceeding with the cleanup.
- Terminate the EC2 worker instance.
- Delete the temporary target EBS volume.
- Delete the snapshot.
- Delete the source root EBS volume.
I hope you find this blog useful. And if you do, please head over to our web site and check out what else that we do to optimize cloud resources. Please feel free to contact FittedCloud support (mailto:firstname.lastname@example.org) if you have any questions.
Other Related Blogs
- A Simple Open Source Tool for EC2 RightSizing Analysis
- Simple AWS EC2 scheduler vs Machine Learning driven EC2 scheduler
- Machine learning CPU Utilization for AWS EC2 scheduling
- A Simple script to implement AWS EC2 TTL (Time to Live)
- AWS EC2 Cost Optimization
- Using Efficiency Metrics to Monitor Public Cloud Costs
- How to shrink EBS root volumes in AWS – with just one click
- How to reduce AWS costs using machine learning driven EBS IOPS provisioning
- AWS Elastic Volumes and FittedCloud EBS Optimizer
- FittedCloud AWS EBS Optimizer for Docker Containers
- An Open Source AWS EBS Cost Analyzer
- Is it possible to use EBS gp2 instead of io1, achieve same performance and save 50%?
- How to optimize AWS EBS using LVM and reduce cost
- How to create thin provisioned AWS EBS volumes and save a ton!
- AWS EBS Performance – Confused?
FittedCloud Cloud Cost Optimization Solutions
FittedCloud offers machine learning based cloud cost optimization solutions that help customers reduce AWS spend significantly. Our current solutions include machine learning driven actionable advisories with click through actions for EC2, EBS, RDS, DynamoDB, ElastiCache, ElasticSearch, AutoScale, Lambda, etc. and full/lights out automation for EC2, EBS, DynamoDB and RDS. Our solution typically can save customers up to 50% of their cost on AWS. For a 14-day Premium free trial, please visit https://www.fittedcloud.com/register-now/.