Businesses spend billions of dollars migrating data between information-intensive applications. Yet up to 75% of new systems fail to meet expectations, often because flaws in the migration process result in data that is not adequately validated for the intended task.
Because the system itself is seen as the investment, any data migration effort is often viewed as a necessary but unfortunate cost, leading to an over-simplified, under-funded approach. With an understanding of the hidden challenges, managing the migration as part of the investment is much more likely to deliver accurate data that supports the needs of the business and mitigates the risk of delays, budget over-runs and scope reductions that can arise.
Whatever the reason for the data migration, the ultimate aim should be to improve corporate performance and deliver competitive advantage. In order to succeed, data migrations must be given the attention they deserve, rather than simply being considered part of a larger underlying project. Without this, and without proper planning, there is a high risk of going over budget, timescales slipping, or even the project failing completely.
Following a structured methodology will reduce the pain involved with managing a complex data migration, but the correct choice of technologies will go a long way to help give the best chance of a successful outcome. It used to be the case that a range of software from different suppliers, plus a lot of technical know-how, was needed to successfully accomplish a data migration, but the architecture becomes difficult to manage, and performance deteriorates due to the number of different interfaces between applications. The ideal solution is a software tool that supports the whole data migration lifecycle from profiling and auditing the source(s) through transformation, cleansing and matching to the population of the target. It needs to be flexible, highly scalable, require minimal technical expertise and be intuitive, so that business and technical staff can work collaboratively. Users should be able to implement complex business rules for data migration or data quality assurance without requiring coding skills or specialist training.