Successful Data Conversion Projects Using Automated Application Mapping

Successful Data Conversion Projects Using Automated Application Mapping

 

The Challenges

 

During application migration, a key success factor is always the effective and correct migration of existing data from the old platform to the new. The two obvious extremes for accomplishing this work are a fully manual vs. an automated approach. While an automated solution is naturally preferred, typically no ‘out-of-the-box’ conversion tools exist are able to handle the complex environments found in practice. Consequently, more often than not, companies fall back to the manual approach. Costs for the design, implementation and use of a throw-away automated procedure typically outweigh the costs of manual conversions.

 

The resulting data conversion challenges can be described as:

 

  • Confirming the scope for the data migration
  • Understanding the ‘core’ data that must be migrated and its inter-relationships
  • Capturing verifiable data consistency rules
  • Establishing a consistent, repeatable and auditable conversion process
  • Automating the conversion process, wherever it is cost-effective
  • Creating test plans to ensure the accuracy and completeness of conversion
  • Reducing business impact and project risk

 

 

Benefits of Automated Application Mapping

 

Application Mapping addresses the challenges described above in three key areas:

First, it brings a robust and time-tested process to the table. All project activities are identified in the plan and can be estimated based on the accurate results of the mapping process, eliminating any guess work.

 

Secondly, the necessary understanding of the data and its relationships can be developed more quickly and effectively with Application Mapping, by viewing the ‘live’ system information. This ensures that the information on which critical decisions are based is complete, current and authoritative (unlike manual reviewing of portions of the data and out-dated documentation).

 

Thirdly, instead of having to develop automated conversion routines from scratch, Application mapping can quickly identify the common patterns inside the application to enable higher degree to conversion routine reusability.

 

 

 

Conversion Framework

 

The following diagram illustrates a sample conversion solution using Automated Application Mapping:

 

data_migration
The shaded area represents Application Mapping’s “black box” conversion solution. The process receives as its input an extract file containing data to be converted (prepared by the client) and produces as its output a load file containing the transformed data ready to be loaded into the target database.

 

The conversion engine also uses as input a series of static (i.e. prepared only once) parameter files that describe mapping specifications for each table, file, column or field that is within the scope of the automated solution. The specifications reference data definition language (DDL) statements that describe the source and target data stores.

 

Repeated Conversion Execution

 

Actual conversion of data is typically planned to take place in segments over a period of time. This helps to mitigate project risks (avoiding the ‘big bang’ approach to data conversion).

 

Automated data conversion can be re-run as often as required with minimal effort. It is still possible that some data will need to be converted manually and entered into the target database via direct database edit. This manual conversion may take place before or after each automated conversion run.

 

leave us a comment, Tell us about your data conversion experience.

Leave a Reply

Your email address will not be published. Required fields are marked *