Open source software underpins the vast majority of public cloud services, and it holds tremendous potential for organizations aiming to create a public cloud-like infrastructure on-premises.
But there’s one big problem preventing that promise from becoming a reality:
The hyperscale public cloud providers have spent enormous resources to develop their own systems and tools to deploy, manage and orchestrate these services, but most enterprises don’t have the resources and expertise to do the same in their own environments. (In large organizations, entire teams can be dedicated to support just one open source project.)
And it all boils down to this: The time and effort it takes to try and do it yourself can significantly detract from the benefits you’re hoping to achieve.
As a result, many open source projects start with a flourish but end up never crossing the finish line to production. In Red Hat’s 2022 “The State of the Enterprise Open Source” report, 32% of respondents listed “lack of internal skills to manage and support enterprise open source” as a barrier to adoption.
Similarly, in the “2023 State of Open Source Report” by OpenLogic, the primary challenges that organizations faced were personnel (experience, skills proficiency) and operational (upgrades, configuration, maintenance, scale, performance, etc.) issues. Clearly, this indicates that organizations need to be prepared to make some investments too.
The importance of this point was recently reiterated by Matthew Leonard, head of storage engineering at Bloomberg, a financial technology organization that has been actively participating in the open source community for the past 15 years. Over that time, the firm has successfully integrated a wide variety of open source projects into its products and infrastructure. For example, Leonard’s team manages part of Bloomberg’s internal private cloud compute and storage infrastructure, which is built on top of cloud native technologies such as OpenStack, Kubernetes and Ceph (a project for which he serves as a member of its board).
“You can’t simply drop an open source project into a large on-prem environment with multiple interdependencies and expect it to work,” Leonard said. “You must go in eyes wide open. The open source software is just a starting point, but you’ll have to continuously invest time and resources. In many cases, our teams also developed some of our own tooling to operate open source at scale in our production environment, which we’ve then gone and contributed upstream to benefit the broader community.”
Let’s dive deeper into these challenges and how Platina Systems can help you overcome them.
Planning
Getting started is often the hardest part of deploying an open source project. What is your objective? It’s of the utmost importance to select a project that meets your specific requirements around scale, performance and availability.
Surface-level functionality is just one of many criteria you need to consider, including who controls the code, the vibrancy of the community and the availability of enterprise-class support.
You have to identify your options, evaluate their GitHub sites and, once you’ve made your choice, figure out exactly what you need to do from there. Most projects start with a simple, “Hello, World”-level example, but how do you move on from that to a large-scale production deployment? What’s the process you need to go through?
A lot of times, there’s no easy answer to those questions.
Configuration
Configuring open source software for your specific use cases can be complex. It’s not just a matter of researching best practices for your configuration parameters and putting them into action.
For starters, understanding (or sometimes even finding) the best practices can be difficult and time-consuming, depending on the project and its level of community support. If and when you do find the best practices, they vary based on your environment.
Your environment and the applications you use create a complex set of dependencies. How do you know which options and settings to use for your particular implementation? A lot of time and effort goes into understanding and testing the corresponding behaviors for your specific environment.
Bottom line: It takes a lot of commitment to even get to the point where you’re ready to configure your software.
Integration
Integrating open source software with existing systems and infrastructure in your production environment poses substantial challenges — especially if you’re running a mix of open source and proprietary software, or if you’ve developed any custom software to support the integration. Many open source projects are controlled or heavily influenced by a sponsoring company, which can make modifying the code to fit your specific needs even more difficult.
All the different components of your environment were implemented in a specific way for a specific reason (for scale, resiliency, robustness, security, ease of maintenance, etc.). The larger the environment, the more complex this becomes, especially for distributed systems. These factors are often outside the scope of the open source project, making integration tricky. You’ll need to make further adjustments to accommodate either in your environment or with the open source software. Could that have unintended consequences up and down the stack?
For example, enterprise networks can be very complex. While your open source project merely requires connectivity, a production enterprise network may be designed for high availability, and connections can traverse different paths, devices or even locations. Then there’s security. Do firewalls need to be considered? Are there policies for encryption, access and other privileges? You’ll need to make the open source software work within the context of these environments.
No two organizations take the same approach, and the larger the organization, the more complex this is likely to be.
Managing dependencies and versioning conflicts is another critical aspect of integration. Every piece of software will have its own libraries and frameworks, each with its own version requirements. Ensuring compatibility and resolving conflicts between these dependencies can be time-consuming and require meticulous attention to detail.
Tuning and Optimization
Nothing is static in production. Everything is always changing. Optimizing your open source software requires continuous tuning.
For instance, different conditions highlight different bottlenecks, such as inadequate memory allocation, bandwidth, I/O, etc. Add on top of this new updates to the open source project, new workloads running on top of your deployment, changes to hardware and many other changes, all of which can also throw a wrench into your performance.
It’s a constant battle to keep your environment fine-tuned and optimized.
Monitoring and Management
Comprehensive system checks, logging, alerting and troubleshooting mechanisms are essential for smoothly operating software in production. But there are so many dependencies in a given environment that it makes these tools difficult to integrate and use effectively, and it often results in home-grown tools being developed..
Managing upgrades, backups and disaster recovery procedures are also vital aspects. But it takes a lot of manual effort to set these up as reliable, repeatable processes in production.
The Platina Approach
There’s no shortage of tools available to help organizations overcome the challenges associated with deploying and managing open source software. But many of these tools are point solutions, and you still have to deploy, integrate and manage them in your environment, just like any other software. It’s a balance where you have to add more to try to lighten your load, and this ends up eating up more and more of your precious time and your valuable resources.
Platina, conversely, is the glue that holds together all these disparate components, taking the work and risk out of deploying and managing best-in-class open source software in real-world, enterprise-class production environments. We solve the tough challenges around configuration, integration, optimization, networking, security, monitoring and more. And you get predictable, low costs and limitless scale in the secure, controlled environment of your choice — either through a managed services provider or in your own data center.
Want to learn more? Request a demo today.
.svg)
.svg)
