Frequently Asked Questions
If you’re new to the DataStax Zero Downtime Migration features, these FAQs are for you.
What is meant by Zero Downtime Migration?
Zero Downtime Migration (ZDM) means the ability for you to reliably migrate client applications and data between CQL clusters with no interruption of service.
ZDM lets you accomplish migrations without the need to change your client application code, and with only minimal configuration changes. While in some cases you may need to make some minor changes at the client application level, these changes will be minimal and non-invasive, especially if your client application uses an externalized property configuration for contact points.
The suite of ZDM tools enables you to migrate the real-time activity generated by your client applications, as well as transfer your existing data, always with a simple rollback strategy that does not require any downtime.
It is important to note that the Zero Downtime Migration process requires you to be able to perform rolling restarts of your client applications during the migration.
In the context of migrating between clusters (client applications and data), the examples in this guide sometimes refer to the migration to our cloud-native database environment, DataStax Astra DB. However, it is important to emphasize that the ZDM Proxy can be freely used to migrate without downtime between any combination of CQL clusters of any type. In addition to Astra DB, examples include Apache Cassandra® or DataStax Enterprise (DSE). |
Can you illustrate the overall workflow and phases of a migration?
Here’s a high-level view:
Here are the phases of a migration from start to finish:
For more, see Introduction to Zero Downtime Migration.
What components are provided with ZDM?
DataStax Zero Downtime Migration includes the following:
-
ZDM Proxy is a service that operates between Origin, which is your existing cluster, and Target, which is the cluster to which you are migrating.
-
ZDM Automation is an Ansible-based tool that allows you to deploy and manage the ZDM Proxy instances and associated monitoring stack. To simplify its setup, the suite includes the ZDM Utility. This interactive utility creates a Docker container acting as the Ansible Control Host. The Ansible playbooks constitute the ZDM Automation.
-
Cassandra Data Migrator is designed to:
-
Connect to your clusters and compare the data between Origin and Target
-
Report differences in a detailed log file
-
Reconcile any missing records and fix any data inconsistencies between Origin and Target, if you enable
autocorrect
in a configuration file
-
-
DSBulk Migrator is provided to migrate smaller amounts of data from Origin to Target
-
Well-defined steps in this migration documentation, organized as a sequence of phases.
What exactly is ZDM Proxy?
ZDM Proxy is a component designed to seamlessly handle the real-time client application activity while a migration is in progress. See here for an overview.
What are the benefits of Zero Downtime Migration and its use cases?
Migrating client applications between clusters is a need that arises in many scenarios. For example, you may want to:
-
Move to a cloud-native, managed service such as Astra DB.
-
Migrate your client application to a brand new cluster, on a more recent version and perhaps on new infrastructure, or even a different CQL database entirely, without intermediate upgrade steps and ensuring that you always have an easy way to rollback in case of issues.
-
Separate out a client application from a shared cluster to a dedicated one.
-
Consolidate client applications, currently running on separate clusters, into fewer clusters or even a single one.
Bottom line: You want to migrate your critical database infrastructure without risk or concern that your users' experiences will be affected.
Which releases of Cassandra or DSE are supported for migrations?
Overall, you can use ZDM Proxy to migrate:
-
From: Any Cassandra 2.1.6 or higher release, or from any DSE 4.7.1 or higher release
-
To: Any higher release of Cassandra, or to any higher release of DSE, or to Astra DB
Here are just a few examples of migration scenarios that are supported when moving from one type of CQL-based database to another:
-
From an existing self-managed Cassandra or DSE cluster to cloud-native Astra DB. For example:
-
Cassandra 2.1.6+, 3.11.x, 4.0.x, or 4.1.x to Astra DB
-
DSE 4.7.1+, 4.8.x, 5.1.x, or 6.8.x to Astra DB
-
-
From an existing Cassandra or DSE cluster to another Cassandra or DSE cluster. For example:
-
Cassandra 2.1.6+ or 3.11.x to Cassandra 4.0.x or 4.1.x
-
DSE 4.7.1+, 4.8.x, or 5.1.x to DSE 6.8.x
-
Cassandra 2.1.6+, 3.11.x, 4.0.x, or 4.1.x to DSE 6.8.x
-
DSE 4.7.1+ or 4.8.x to Cassandra 4.0.x or 4.1.x
-
-
From Astra DB Classic to Astra DB Serverless
What challenges does ZDM solve?
Before DataStax Zero Downtime Migration was available, migrating client applications between clusters involved granular and intrusive client application code changes, extensive migration preparation, and a window of downtime to the client application’s end users.
ZDM allows you to leverage mature migration tools that have been used with large scale enterprises and applications to make migrations easy and transparent to end users.
What is the pricing model?
The suite of Zero Downtime Migration tools from DataStax is free and open-sourced.
Is there support available if I have questions or issues during our migration?
If needed, ZDM Proxy and related software tools in the migration suite include technical assistance by DataStax Support.
For any observed problems with the ZDM Proxy, submit a GitHub Issue in the ZDM Proxy GitHub repo.
Additional examples serve as templates, from which you can learn about migrations. DataStax does not assume responsibility for making the templates work for specific use cases.
Where are the public GitHub repos?
All the DataStax Zero Downtime Migration GitHub repos are public. You are welcome to read the source and submit feedback via GitHub Issues per repo.
-
ZDM Proxy open-source repo: in addition to sending feedback, you may submit Pull Requests (PRs) for potential inclusion.
-
ZDM Automation repo for the Ansible-based ZDM Proxy automation.
-
Cassandra Data Migrator repo for migration of larger data quantities and where detailed verifications and reconciliation options are needed.
-
DSBulk Migrator repo for migration of smaller data quantities.
Does ZDM Proxy support Transport Layer Security (TLS)?
Yes, and here’s a summary:
-
For application-to-proxy TLS, the application is the TLS client and the ZDM Proxy is the TLS server. One-way TLS and Mutual TLS are both supported.
-
For proxy-to-cluster TLS, the ZDM Proxy acts as the TLS client and the cluster as the TLS server. One-way TLS and Mutual TLS are both supported.
-
When the ZDM Proxy connects to Astra DB clusters, it always implicitly uses Mutual TLS. This is done through the Secure Connect Bundle (SCB) and does not require any extra configuration.
For TLS details, see Configure Transport Layer Security (TLS).
What are the benefits of using a cloud-native database?
When moving your client applications and data from on-premise Cassandra Query Language (CQL) based data stores (Cassandra or DSE) to a cloud-native database (CNDB) like Astra DB, it’s important to acknowledge the fundamental differences ahead. With on-premise infrastructure, of course, you have total control of the datacenter’s physical infrastructure, software configurations, and your custom procedures. At the same time, with on-premise clusters you take on the cost of infrastructure resources, maintenance, operations, and personnel.
Ranging from large enterprises to small teams, IT managers, operators, and developers are realizing that the Total Cost of Ownership (TCO) with cloud solutions is much lower than continuing to run on-prem physical data centers.
A CNDB like Astra DB is a different environment. Running on proven cloud providers like AWS, Google Cloud, and Azure, Astra DB greatly reduces complexity and increases convenience by surfacing a subset of configurable settings, providing a well-designed UI known as Astra Portal, and a set of APIs to interact programmatically with your Astra DB organizations and databases.