Customer Case Study
dataxu Cages Untamed AWS Costs using FittedCloud
Customer Case Study : dataxu
Inc. 5000 Hall of Fame member dataxu provides software-as-a-service that helps marketers better understand their customers. Every second, the dataxu Marketing Cloud processes over 1.5 million ad requests, 3 million bid requests, and makes more than 40 billion decisions to select and bid on ad impressions that are most likely to convert. But that scale comes at a cost.
The platform largely runs on Amazon Web Services (AWS), resulting in bills of hundreds of thousands of dollars every month. Contributing to the costs are thousands of EC2 instances and multiple petabytes of data. As dataxu continues to be one of the fastest-growing companies in America, the cloud infrastructure spending grows and consumes budget that could be spent elsewhere.
“With a large portion of our IT budget going to AWS, we are always looking for ways to shave stack costs,” explains Scott Bruce, Director of Platform Engineering at dataxu.
At the beginning of the year, dataxu management wanted to improve the company’s balance sheet and asked Scott to focus on increasing cost efficiency. “We looked at all kinds of options: renegotiating licenses, dropping products,” says Scott, “and if we could cut waste even by one percent, it would have a big impact.”
dataxu had already implemented a 3rd party Cloud Management tool that provided reports on resource utilization and costs but it fell short. “I checked weekly and monthly reports to find trends, but it’s almost too late at that point.” Development teams are spinning up new instances all the time and a few weeks of wasteful computing quickly adds up. While the monitoring tool also claims it could ‘right-size’ instances, “it’s not much better than a report,” cautions Scott.
After Scott analyzed those reports to identify a problematic resource he had to spend significant time manually figuring out the cause and the right solution — what was the appropriate IOPS for the EC2 instance? What’s the right storage capacity? Was it being used at all? Scott realized this approach didn’t scale and sought out other options.
(Data shown is not related to dataxu)
“I thought about creating our own tool using AWS APIs,” Scott reported. He wanted a tool that identified inefficient resources accurately, quickly, and proactively notified his team. But the time to build and maintain such a tool would be even higher than the time spent manually tracking down and fixing waste. Scott set about finding a tool that solved his problem, which is how he learned about FittedCloud.
“FittedCloud is what I would have built if I had the time,” declares Scott. It monitors and collects utilization for cloud resources, provides actionable notifications, and eliminates waste with a single click. Machine learning is used to predict resource utilization based on past usage and can dynamically resize EC2 and EBS without disruption to optimize costs.
Getting started with FittedCloud was seamless, with IAM security configuration being the only area that required bit of thinking. “We didn’t put much effort into it,” Scott confessed.
Within a few hours, FittedCloud provided dataxu 644 recommendations that could save nearly $200,000 per year. With a few days of machine learning, FittedCloud recommended additional advisories with more potential savings per year. Each advisory shows the potential savings and exactly what changes to make to optimize costs, such as resizing instances at a particular time, instance stop/start schedules, and re-provisioning IOPS.
The recommendations are sorted with the greatest cost savings listed first so Scott finds that “it’s convenient to pick off the worst offenders quickly, and tell them not to do it again.” The biggest waste tends to be instances that are abandoned, instances that are too big, or storage provisioned with too many IOPS.
Scott’s favorite part is how actionable and accurate FittedCloud’s email notifications are: “Every problem it identifies is a real issue that needs to be fixed, and you don’t have to think about what to do — it’s right there in your face with a fix.” It’s a big difference from AWS Trusted Advisor, where somebody must remember to go look at it, and then figure out what to do.
So far dataxu has used FittedCloud to identify and resolve issues that would cost $120,000 per year if left unresolved. Scott is working through the FittedCloud advisories as he has time and plans to allocate more resources from his team to plug away at the remaining advisories.
Beyond using actionable advisories, Scott also plans to consider using full optimization automation capabilities that FittedCloud offers for EC2, EBS and DynamoDB in the future.