Avoiding SLA Breaches with Zero Downtime Database Migration using Arcion

Who

SegMetrics is an analytics platform that helps digital marketers understand the full customer journey from click to conversion. Its platform provides marketers with the insights they need to optimize their campaigns and improve their bottom line. It allows marketers to track their campaigns and measure their results based on full-funnel attribution and customer lifetime value.

SegMetrics is a remote-first company with over 10 years of experience in the industry. It partners with some of the top marketers in the world, including Eben Pagan, 7 Mile Media, Entrepreneur on Fire, Dr. Axe, among many others.

What and Why

SegMetrics is trusted by several world-class marketing teams to serve them with the most accurate and real-time data including native imports from various marketing tools and website browser tracking. To meet the evolving needs of their customers, Keith Perhac, the CEO and founder of SegMetrics, and his team realized the need to migrate their customer data from DigitalOcean to SingleStore. But they were staring at 4TB of business critical data. The sensitivity of handling customer data and its business critical impact mandated that they choose a robust solution, which ensures minimal downtime and avoids all kinds of data synchronization challenges.

Before turning to Arcion, Keith and his team had explored alternative approaches. One option was to implement a one-time manual migration of a snapshot of the data at a specific point in time, followed by a final migration of the "latest data." However, due to the structure of their data (which involves millions of UPSERT operations an hour), they realized that completing the migration within a reasonable timeframe would be challenging. Such an approach would cause ~32 hours of downtime. Their customers who rely on real-time data to maintain their competitive edge and serve their end customers better expect minimal delays. This manual process would have impacted their end clients, and gone against their service level agreements (SLA).

There were additional risks in the manual process. As a one time migration, once the changes were made, those changes would be irreversible. They would be unable to roll back the deployment to address any unforeseen issues - typical to database migrations of large data volumes.

So, the team needed a solution that was proven and one that could guarantee them minimal downtime while offloading the risk of unsynchronized data between the source and target databases.

How

It was right about that time when they found out about Arcion.

As the SegMetrics team was actively exploring alternatives, a trusted friend from their business circle recommended Arcion as a potential solution. But the highlight was when SingleStore shared about their partnership with Arcion. They knew that they had found a match.

After a week of testing and trialing the solution with single-import and continuous migration tests, the SegMetrics team was convinced about the robustness and performance of Arcion Cloud. By using Arcion Cloud, they eliminated any infrastructure setup requirements. The simplicity of the Arcion solution coupled with the convenience provided by a cloud-offering sealed the deal for them.

At Arcion, we were excited to help SegMetrics with their goals and so was their team to start using Arcion to migrate their data to SingleStore. However, we quickly ran into some challenges.

First, the configuration options for DigitalOcean's managed databases were not compatible with Arcion. This caused some errors during the setup process.

Second, a large amount of data was continuously being pushed into SingleStore's database. This put a strain on both the reading and writing in the database and caused some performance issues.

It took about three weeks troubleshooting these issues and fine-tuning the configuration. The SegMetrics team also conducted migration tests with the data to ensure that the import delay remained below 10 minutes. When we were finally ready for the migration, the whole process took only ~12 hours to complete. And because of Arcion’s real-time Change Data Capture capability, the SegMetrics team was able to achieve ongoing synchronization between MySQL on DigitalOcean and SingleStore after the migration was complete. They were able to make sure that the data was up to date when they switched over.

"The ongoing synchronization is honestly the killer feature for us. Because we deal with around 50GB of UPSERTED data every day, we actually performed the full migration a dozen times as we were making updates to the SingleStore database to make sure that we were getting enough performance out of the systems. During one performance test, we were able to transfer 110 Million rows in under 4 minutes."

Keith, CEO SegMetrics

SegMetrics completed a real-time migration, kept the data in sync, moved the app over to the new SingleStore database, and then with a single line of code in their config file, the SegMetrics team could swap between either using SingleStore or DigitalOcean for their data.

This enabled them to do a feature-flag release, where they could slowly scale the SingleStore data queries up, and make sure they weren’t having any issues during the migration. And even if they needed to make large changes, they could simply drop the whole database and re-do the migration.

Alongside the Arcion solution’s core capabilities, what really stood out was the close collaboration between the Arcion and SegMetrics teams. Keith and his team were convinced that they could count on Arcion if things didn't go as planned.

Large-scale data migrations are often challenging. Modern businesses like SegMetrics look for minimal downtime solutions and one that guarantees business continuity at all times. “Reach out to Arcion for a quick chat that could shave hours off of your migration”, said Keith, “they'll be able to help you plan your migration, troubleshoot any issues, and make sure that everything goes smoothly.”