Migration Projects That Ignore Data Quality Are More Likely to Fail, Overrun or Have Poor Adoption Due to Lack of Confidence in the Data

12 November 2007

Poor data quality is one of the biggest stumbling blocks in data migration projects, and IT professionals fear migrations so much they’ll do just about anything to avoid it, according to data quality specialists, Datanomic Ltd. The IT department may know all there is to know about database and application design, but it is clear that they do not know how the information in these systems is used and what it actually means to the business. Many application migration projects seek to rule data quality out of scope. With cleanliness and complexity of data so poorly understood, a high proportion of systems migrations are seen as failures by the business community.

A recent survey conducted by Celona Technologies, revealed that 95% of 212 IT professionals surveyed said that they fear migrations. The fear of delivering late, or failing to get all of the data migrated were each cited as a worry by more than a third of those involved in the survey. That’s bad enough, but, if you’re about to embark on a migration yourself, it will be of little comfort to know that a similar percentage (33%) worry that they might never finish the migration project at all.

“Businesses worldwide spend significant sums of money migrating data between information intensive applications, whether into new systems or as regular feeds to existing ones,” said Dr Jonathan Pell, CEO of Datanomic. “Often the system itself is seen as the investment, whereas any data migration effort is seen as a necessary but unfortunate cost. This can result in an over-simplified and under-funded approach to the migration that can lead to delivery of data that is not fit for the purpose intended. This undermines the performance of the target system, damages business and user confidence in its capabilities and means the expected investment returns are not realized.”

It appears that many IT professionals find the thought of a migration so frightening that they’ll do anything to avoid it. The Celona survey found that 59% of respondents said that they had been so discouraged in the past that they have abandoned a migration completely. The drivers for these fears seem to be concerns about the lack of understanding of the information that needs to be migrated – 126 of the 212 respondents blame their fears on ignorance about the complexity and cleanliness of data, whilst nearly 90% believe that data complexity is continuing to increase.

“Projects that do not address the data challenge from the outset, face a high probability of over-running, incur additional costs and expose the target system to the risk of poor user adoption, due to a lack of confidence in the data it contains,” added Pell. “It would be unfair to see this as a problem for the IT departments making. They may be the guardians of the CRM system, but data is a business asset and the databases and applications it resides in are merely repositories for that asset. Business users must play an active role in any application migration, not only in the selection and sign-off of the new system but at every stage throughout the data migration lifecycle. To prevent a migration becoming a leap in the dark, organisations need to start by understanding their data thoroughly, and insist on an Information Quality Assessment from the outset.”

To address these issues, Datanomic’s dn:Director takes a holistic approach to data quality management enables business and technical users to work collaboratively. Described by industry analysts as “arguably the most flexible data quality product”, Datanomic’s dn:Director provides a single, unified platform that enables non-IT personnel, such as data stewards, business analysts and owners of data, to identify and rectify the compromised data which place enterprise operations at risk. As a single product that encompasses profiling, analysis, transformation, cleansing and matching, dn:Director 6.1 enables better understanding, improvement, protection and control of data quality for all types of structured and non-structured, real-time and non-real time information.

Become a bobsguide member to access the following

1. Unrestricted access to bobsguide
2. Send a proposal request
3. Insights delivered daily to your inbox
4. Career development