Data migration is the selection, preparation, extraction, transformation and transfer of quality data. This blog post will give you a preview of the migration process and offer some tips for preparing, collaborating, and executing.

There is no dispute: moving data from one platform to another is a complex process. Data migration teams and data processing teams must speak the same language and share the goal that the migration gets done right the first time. But in reality, it’s usually the technical team that must learn how to communicate with non-technical staff and make sure that nothing gets lost in translation.

Here are a few data migration guidelines, or steps, for teams to keep in mind when migrating to a new case management software. These are the high-level check points.

• The Team Roles
• Data Profiling
• Clarification of Gaps
• Data Quality Rules
• A Migration Plan

 

Ensure The Teams Understand Their Roles In Data Migration

There is a technical gap that the migration team must close to make things work properly. While there are plenty of technologies and tools for physical extraction, transformation, and loading of data, people too often view data migration as a purely technical exercise.

In reality, only the consumers and owners of the data understand its meaning and can define the value it must retain. It takes effort by both teams, working together, to achieve this end. Here are two key facts important to remember.

• Data migration is a business issue, not a technical one. The data owners are the experts in a business process. They’ve been running the legacy systems and best know the flow of information. They have the expertise to make judgments about the quality and usefulness of the data.
• The data migration analysts can’t know more about the business rules than the data owners. It is the job of the technical team to learn as much as possible about the business processes. That way, the technical team can better ensure that the migration procedure does not compromise business rules or data flows.

Understanding this distinction and acknowledging the importance of the data owners are fundamental to the data migration process. Now you’re ready to move to the next step.

 

Perform Extensive Data Profiling

The data owners know the location of the data sources. Early identification of all data sources is crucial to accurate estimating of timescales.

Departments often create “mini databases” which support their everyday activities in working with the main database. These mini databases usually patch business process gaps in the “official” system. Some may be unacknowledged, even though they are critical to the department’s processes. Each can be in a different format and of varying quality, making it difficult to “link” them together.

Once the legacy data stores have been identified, both teams can agree on the scope of work.

At this stage, the teams should focus on the data stores. The data owners, on discovering and cataloging the data stores and their relationships to one another. The technical team, on examining the data stores to understand how they work, the data they hold, and the challenges they pose.

Finally, both teams need to determine what data to migrate. In an ideal world, this would be “everything.” The reality, however, is that some data does not add value or may be covered by another source. For example, those mini databases might create more noise than data quality, and further complicate the data migration. For this reason, the technical team must work closely with the data owners to differentiate between value and noise.

 

Analyze And Map the Gaps

During this step, both teams work together to analyze and quantify data gaps.

Gap Analysis

A legacy system can accumulate various issues that need resolution. The figure below illustrates a few examples.

Some gaps will be familiar to the end users, others will surface only under pre-migration database examination. Make it a critical task to “flush out” all issues as soon as possible and to promptly take the proper action.

Mapping

Mapping is the process of connecting fields in the legacy data stores to fields in the target system. The use of a spreadsheet is perfectly acceptable for this activity. Mapping is effective only through close collaboration between the technical and business teams.

 

Define Data Quality Rules

Data quality rules measure the quality of the data and define the action for fixing or mitigating quality issues. Each data quality issue should be assigned a rule. The table below is a basic but working model for this step.

 

Create A Migration Plan

Even though data migration is a major activity in the overall project plan, the main migration activities should be mapped separately to a migration plan with an agreed-to schedule.

Use the plan as the official agreement between the technical team and data owners on the implementation approach. Most migrations will conform to one of three approaches.

• Big Bang – All the data is moved in one go. It is the most common approach.
• Phased – The data is moved in separate chunks, perhaps defined by business or location.
• Parallel – The data is moved to the target. The legacy data remains intact and continues in use. Changes to legacy data are synchronized to the target.

 

Execute The Migration Plan

Migration software – also known as the extract, transform, and load (ETL) tool‒is used to load the data into the target system. The technical team can create an in-house tool or use a third party tool. Whatever the source, migration software should do more than the three primary tasks. A solid ETL tool will also do these key functions:

• Synchronize data, if the source data must be used during the migration, changes in the source can be updated in the target database
• Reformat the data and combine from multiple sources
• Read the data from the legacy database
• Write the data to the target database
• Provide reports on execution
• Start and stop the process
• Provide an audit trail
• Manage data errors
• Validate the data Wrap-up

Many organizations identify data migration as the biggest roadblock in the move from a static legacy system to a dynamic case management system.

If your organization is facing such an obstacle, there are realistic ways to clear the path. Both the data owners and the technical team together will need to work through the business rules, map out all the databases that make the business rules work, and create an execution plan for converting all the disjointed databases into a unified, purified database… without loss of process-critical data.

It’s a daunting task. But the good news is that switching from an old platform to new case management system is now easier than ever. There are plenty of case management platforms to choose from. All it takes is a flexible platform and an experienced team that knows case management inside and out. Finding the necessary technical expertise can be a challenge. Hopefully, now you can better evaluate teams and pick the right solution to meet your data migration needs.