During application migration, a key success factor is always the effective and correct migration of existing data from the old platform to the new. The two obvious extremes for accomplishing this work are a fully manual vs. an automated approach. While an automated solution is naturally preferred, typically no ‘out-of-the-box’ conversion tools exist that are able to handle the complex environments found in practice.Consequently, more often than not, companies fall back to the manual approach. Costs for the design, implementation and use of a throw-away automated procedure typically outweigh the costs of manual conversions.Data conversion challenges can be summarized as follows:
Confirming the scope
Understanding the ‘core’ data that must be migrated and its inter-relationships
Capturing verifiable data consistency rules
Establishing a consistent, repeatable and auditable conversion process
Automating the conversion process, wherever it is cost-effective
Creating test plans to ensure the accuracy and completeness of conversion
Repeatability to support phasing and testing
Reducing business impact and project risk
Benefits of using Mapador
Mapador addresses the challenges described above in three key areas.
First, it brings a robust and time-tested process to the table. Using Mapador’s flexible, quality-driven, self-auditing and repeatable process, the data migration project can be managed effectively and efficiently. All project activities are identified in the plan and can be estimated based on the experience of other Mapador data migration projects. The repeatability inherent in such a controlled process enables regression testing to be done at any point in the conversion, thus streamlining the elimination of any data conversion errors. This results in a higher overall conversion quality than is attainable with an ad-hoc process.
Second, necessary understanding of application data and its relationships can be developed more quickly and effectively, by viewing production (‘live’) system information captured in the Mapador Repository. This ensures that information on which critical decisions are based is complete, current and authoritative.
Third, data migration projects can be delivered without having to develop automated conversion routines from scratch. Mapador’s application understanding functionality is accompanied by a targeted and customizable set of data migration tools. The ready-made capabilities within Mapador ensure that the conversion routines are simpler and that the cost of automation is justified. The open architecture allows the tools to accomplish client-specific tasks, as if the tools were made for the specific data migration at hand. As a result, more conversion processes can be automated, eliminating error-prone and time-consuming manual conversions.
Conversion Framework
The diagram illustrates a sample conversion solution using Mapador.
In the diagram the shaded area represents Mapador’s “black box” conversion solution. The process receives as its input an extract file containing data to be converted (prepared by the client) and produces as its output a load file containing the transformed data ready to be loaded into the target database.
The conversion engine also uses as input a series of static (i.e. prepared only once) Excel spreadsheets that describe mapping specifications for each table, file, column or field that is within the scope of the automated solution. The specifications reference data definition language (DDL) statements that describe the source and target data stores.
Repeated Conversion Execution
Actual conversion of data is typically planned to take place in segments over a period of time. This helps to mitigate project risks by avoiding the ‘big bang’ approach to data conversion.
Automated data conversion can be re-run as often as required with minimal effort. To ensure perfection, Mapador’s architecture allows for direct manual intervention during the otherwise automated process.
Mapador’s data and application migration processes are tightly integrated to ensure timely execution, better resource allocations and overall success.
