While the cloud may be an efficient option for a startup with a great idea and a few team members equipped with their laptops, their cloud dependency becomes a ubiquitous part of everyday life. As a company grows, so does the cloud service bill growing alongside it. Moreover, as their cloud dependency balloons, the more likely it is that the public cloud becomes increasingly less cost-effective.
In fact, for many data-heavy workloads, the public cloud becomes an entangled web of hidden costs that continue to grow throughout business-critical environments as the massive amounts of data flood their servers.
For a visual to understand data-heavy workloads, in the case of UPS trucks, sensors are used to collect data on roughly 200 different environmental and operational factors. Their delivery fleet consists of roughly 110,000 vehicles with each one recording 200 different types of data which equals 22,000,000 individual data points per day.
Now, today, when UPS wants real-time visibility into their operations using this sensor data, they decide to transmit data from the trucks every six seconds - that makes 14,400 uploads per day, generating over 300 billion unique data points in a single day.
How many other IoT applications do you think UPS will intelligently devise in the next year? Now equate that number to exponentially increasing cloud service costs that were not planned for when the organization was looking solely at the 22M data points per day.
A few years ago, sensors, the Internet of Things (IoT), fifth-generation wireless technology (5G), and computing at the edge barely existed. Now, enterprises need to capitalize on these technologies to compete.
Organizations are all too often caught by surprise when it comes to the actual price of cloud computing in the age of big data. An IDC report from 2018 shared that 80 percent of enterprises that had migrated to public cloud began a secondary migration (back) within two years after they realized that the cloud was not all it was cooked up to be.
Here are several hidden public cloud costs that make the cloud journey for many organizations...well...cloudy.
Hidden Cost #1: Data Transfer Costs
It wouldn’t take a rocket scientist to correctly calculate data transfer costs when setting up a cloud server deal, but a team of them managed to do just that in 2019.
When NASA started to plan for its future data storage needs, the logical conclusion was to expand its cloud contract. The agency calculated it would need 215 additional petabytes of storage by 2025, a figure they used to estimate expected fees. All told, the impressively smart scientists calculated they’d pay their cloud provider around $65 million per year.
As it turns out, even impressively smart scientists can miss things sometimes.
It couldn’t have been a great day around the office when NASA’s Inspector General revealed the results of a routine audit. The initial $65 million estimate was way, way off. In fact, the agency will owe its cloud provider $95 million per year to hold its Earthdata Cloud. In addition to the cloud costs and egress charges, they also are still responsible for the costs to operate their existing Distributed Active Archive Centers (DAACs) or data centers.
What the NASA team failed to take into account were data egress charges. In other words, if scientists want to actually use the data NASA is collecting, the agency will have to pay every time they download it.
How could a group of some of the smartest people in the world miss $30 million worth of data transfer charges? It’s not that Cloud providers hide these charges, but the methodology is complex and a bit vague. Moving your data from the cloud, or enabling your customers to pull down data, is not free on any of the major cloud providers.
Cloud providers can't predict how your app will be used and they won't tell you when you are grossly exceeding estimates (if you have even tried to calculate them). Even if you have careful predictions, you would need to have instrumented controls within your internal systems to enforce them. Which, of course, adds up to more costs in labor and resources.
Egress data transfer charges can add up to an unwelcome surprise.
Hidden Cost #2: Cloud Utilization Costs
Public cloud providers charge a fee every time a company attempts to access its data. While the fees are tiny—sometimes fractions of pennies per hour—cloud providers are banking on high volume.
That volume comes in the form of the millions of tasks companies perform every day to read, write, modify, analyze, and transmit data every second of every day. Those costs add up, and they grow alongside a company as it scales. A public cloud solution that was affordable in the early days may not make sense, transactionally, in the longer term.
To paint the picture, Adobe developers unintentionally racked up $80,000 a day in charges for a computing job that ran on Azure and wasn’t discovered for more than a week.
The Adobe example highlights an important aspect of enterprise cloud management: cloud resources must be monitored very carefully and consistently. It is not a set it and forget it type process. If an organization leaves a VM running without dedicated oversight, it will be caught by a cost surprise. To use these resources cost-effectively, organizations need to adopt a change in how their cloud resources are utilized as well as a dedicated commitment (and extremely knowledgeable team) to closely monitor, provision, and maintain their unique cloud environments.
Hidden Cost #3: Migration Costs
While moving to the cloud can lead to many positive gains for a growing company, the cost of the initial transition is not one of them.
In a survey conducted on 250 IT managers, it was reported that “More than 70 percent of companies that moved to the cloud were not aware of these costs following adoption. Lack of vendor transparency, continuous maintenance costs, and lack of cloud expertise play a role in unpredicted costs and implementation problems.”
IT teams need to not only train up existing staff, including staff and management in other departments, but migration may require expensive new tech expertise and cloud management. All while continuing to operate their existing infrastructure.
The cloud, like any other tool, must be well understood and optimized for the advantages it brings. Along with experts running your private datacenter, you now also need to hire cloud managers for the ongoing monitoring of how applications operate, how cloud resources should be optimally configured, aggregating multiple copies of data, shutting down application-testing servers at night, and beyond.
For many, this has resulted in an indefinite number of years that a dual infrastructure is required.
A recent article from Tom Nolle, President of CIMI Corp, supports the notion that cloud computing isn’t necessarily cost-saving. According to users, only a fifth of these applications “clearly met the business case” for the cloud. Another 40% were “marginally” justified, and the remaining 40% “failed to meet the business case”. About a third of these were then migrated back in-house.
The economics of each type of cloud project is unique, and while most assessments of cloud costs versus data center costs fail because they’re not leveling the playing field in terms of assumptions, most cloud projects are poorly executed, to begin with, and a poorly executed project creates bad results.
IT teams managing existing applications will not be able to quickly shift to running it in an entirely new environment overnight. An investment in your workforce, third party services, training, and additional hardware and/or software may become a necessity depending on the type of move you are making - whether it be new clusters for new applications, a complete lift and shift, cloudbursting, or other hybrid cloud scenarios.
Often, the fundamental work style and enterprise mindset will need to shift too. Just about every process, from project management to customer service, may need to change.
Hidden Cost #4: Business Process Reengineering Costs
Sometimes, organizations expect a cloud migration to improve efficiency and functionality across the board. The truth is that any issues that existed before a cloud migration will continue on the cloud. While the initial transition may be complete, the ongoing business changes needed will continue to surface and drive more spending.
For example, in the case of a complete lift and shift of an application to the cloud, Mark Thiele, Technical Standards Chairman for the International Data Center Authority says, “Lifting and shifting an application to cloud exposes all the gaps in understanding, links, security, and performance issues in ways that are difficult to understand prior to the move.”
Large-scale, established enterprises may have little trouble adjusting, but for companies with immature processes, the move could incur enormous impactful costs.
Touching back to the recent paper from the Journal of Cloud computing, survey respondents reported that they:
“Had to change some processes to move to the cloud and they still wish to change some more, but it is not a one-day task [...] This step of business process reengineering is largely, naïvely-underestimated despite being complicated and vital for a cloud project’s success. The complication around and the importance of process reengineering arise from a number of reasons. One is that responsibility and process ownership is often vague.”
A Senior Technical Architect recently wrote about his experience working on a cloud migration project for a client and cited 10 post-migration costs that need to be considered including:
- Monthly/Yearly infrastructure and software (licensing/support) costs
- System maintenance and operation costs
- System updates
- Software version updates and patches
Unfortunately, vendors often gloss right over this crucial aspect of migration.
The Private Cloud Cost-Savings Advantage
For organizations considering a move back “home” and frustrated with the cloudy costs of the public cloud, a private cloud has advantages the public cloud can’t match when it comes to controlling spiraling costs. You have the ability to leverage your choice of hardware without vendor lock-in, decoupled, distributed storage, and disaggregated delivery.
The rub: how do you even begin to manage it all on your own having only relied on the cloud provider services for so long?
Enter Platina. You might be born in the Cloud, but it doesn't mean that’s where you have to stay. You can have all of the benefits of Public Cloud (such as APIs, Containers, and Storage) on-premise with Platina Systems.
Platina builds, automates, and manages your private and hybrid cloud environments from the bare metal layer up. Empowering you to deploy incredible applications faster, with fewer resources, and at a lower cost.
Learn more about how we convert your chosen infrastructure stack into a uniformly managed, workload-ready, cluster of resources that, on average, saves organizations 50% in infrastructure costs.