Making Persistent Storage for Production Kubernetes Attainable
Containers have become highly popular among DevOps shops due to their advantages in the areas of agility, simplicity and scalability. These same qualities make them attractive candidates for hosting modern applications such as Cassandra, MongoDB, Apache Hadoop and Spark in production. One of the most significant challenges when it comes to shifting containers from test and development to production use cases is the issue of persistent storage for stateful containers. The use of legacy storage platforms for providing persistent storage for containers adds complexity, because it creates multiple infrastructure and management silos. In addition, the traditional approach to data management infrastructure built from multiple storage systems and tools does not allow for shared storage, application mobility or unified data access – all concepts that are inherent to Kubernetes.
Introducing the Reduxio Magellan Cloud Data Platform
For its part, storage provider Reduxio has shifted its focus from providing a hardware-based appliance for VMware-based virtualization environments, to focus exclusively on enabling production Kubernetes container-native storage environments. Reduxio’s new Magellan Cloud Data Platform (which will be generally available in the fall of 2019) is fully software-defined (in that there are no dependencies on the underlying hardware) and runs natively on Kubernetes. Magellan is a microservices-based, container-native storage solution – as opposed to a monolithic storage stack – deployed in a container. As a result, Magellan allows independent scalability of capacity and performance for greater cost-effectiveness. It also makes applications and data mobile across clouds, while facilitating unified data management.
Magellan’s microservices architecture is based on a Reduxio-developed metadata architecture that completely separates storage volumes from the data that belongs to those volumes. This allows the stack to be disaggregated into separate microservices for metadata management, application interfaces, and storage media management. Different types of media are supported, with dynamic continuous tiering allowing Magellan to provide multiple classes of storage for various types of applications. Currently, a container storage interface (CSI) plug-in allows persistent containers to provision volumes that are accessed over NVMeOF or NVMe/TCP. Reduxio has plans to add other methods to access data – including object – in the future. Magellan supports any Kubernetes distribution on bare metal infrastructure, virtualized infrastructure, or on a public cloud service.
Improving Application and Data Mobility
Magellan’s disaggregated, microservices-driven approach provides a number of benefits when it comes to application and data mobility as well as data management. With traditional application migration, when persistent data is installed, all the data in all the volumes for the application must be copied before the application can be restarted in the new location. The time to complete this data copy depends on multiple factors including the amount of data involved and the network bandwidth available. That being acknowledged, typically an application needs to be taken offline for a significant amount of time in order to complete the data migration process. Magellan’s metadata data architecture allows a volume to be moved between clouds or data centers instantly, according to Reduxio – enabling the application to be migrated equally as fast. The application and its volumes can be moved at the same time and the application can be started on the new infrastructure without delay, while data is being copied.
Reduxio also employs a number of strategies to minimize the amount of data that needs to be copied. Write operations are always done to the new location and global deduplication is used to identify common data, while smart prediction and pre-fetch capabilities are used to prioritize the copy of hot data first to minimize read operations from the source.
In addition to application migration, enhanced data mobility can be used for a variety of use cases in edge, multi-data center, hybrid cloud, and multi-cloud deployments. For example, instant incremental sync can be used to refresh volumes across clouds for a broad set of use cases including distributed continuous integration (CI) and continuous delivery (CD) (CI/CD) pipelines. Global deduplication and compression also reduce costs related to copying data from public clouds.
Retaining Production-Grade Robustness and Data Protection
Reduxio has always focused on supporting business-critical applications with data protection as a key differentiator – specifically facilitating near-zero recovery point objectives (RPOs) and recovery time objectives (RTOs) to minimize data loss and downtime. Reduxio retains that focus with Magellan. According to Reduxio, Magellan supports one second-granularity data recovery as well as clones and asynchronous replication with instantaneous data recovery. In addition, erasure coding is used for data resiliency, avoiding the overhead of mirroring, while global inline deduplication and compression further maximize the utilization of available storage.
Additional Benefits of a Microservices Architecture
The Magellan platform’s microservices architecture enables scaling of performance and capacity independently and rapidly (to thousands of nodes and petabytes of data, according to Reduxio). This stands to increase cost efficiencies while the disaggregation of the platform also means forklift upgrades are not required since individual microservices can be updated to deliver new features. The platform is extensible since new microservices can be added for new capabilities and to support new technologies, without needing to take applications offline.
StorageSwiss Take
It is clear that containers will play an important role in hosting mission-critical applications in the data centers of tomorrow. What remains to be seen is how quickly that shift will happen. Reduxio is well-positioned to help enterprises accelerate the shift to a container-based infrastructure. Its key differentiators are data mobility and the focus on providing enterprise-grade capabilities required by business-critical applications in container environments. These include, as previously noted, long-standing competencies around data protection. Another way in which Reduxio is playing to its strengths is its microservices architecture, enabled by its metadata architecture, which facilitates more efficient scaling of the storage environment.
Key use cases for Reduxio’s Magellan platform include providing cost-effective and elastic capacity and performance alongside end-to-end data management for modern database and data processing applications. Additionally, rapid cloning and improved data mobility can help to scale CI/CD pipelines across multiple clouds. We also see an opportunity for the maturing artificial intelligence (AI) and machine learning (ML) workloads to accelerate training and inferencing, and to manage large amounts of data in a more streamlined and consistent way.
Senior Analyst, Krista Macomber produces analyst commentary and contributes to a range of client deliverables including white papers, webinars and videos for Storage Switzerland. She has a decade of experience covering all things storage, data center and cloud infrastructure, including: technology and vendor portfolio developments; customer buying behavior trends; and vendor ecosystems, go-to-market positioning, and business models. Her previous experience includes leading the IT infrastructure practice of analyst firm Technology Business Research, and leading market intelligence initiatives for media company TechTarget.
StorageSwiss.com - The Home of Storage Switzerland 