In this conversation with Christine Liu, we gain deep insight into the processes associated with data conversion. In addition. we meet Christine Liu who has extensive experience leading data conversion projects.
Q: At a high level, will you please define what data conversion is?
Christine: Data migration and data conversion are often used interchangeably. Data migration is a complex process of selecting, preparing, extracting and transforming and permanently transferring data from one format to another or one application/system to another. My job as a data consultant is to convert data from legacy systems to Costpoint for either new implementations or mergers/acquisitions. Data conversion also covers a process called data restructure where the data is modified in an existing database. The Data restructure can often occur in a merger and acquisition scenario where data needs to be modified to fit within a new corporate structure. A simple example is the modification of Employee IDs.
Q. Will you please differentiate “mapping” from “crosswalk”?
Christine: People often use these terms interchangeably, but I draw a clear distinction between the two. Mapping is all about where data is coming from and then where it is going to. For example, if you are mapping from JD Edwards to Costpoint – you need to know from what column in which table the data is coming (in JD Edwards) and then what is the equivalent table and column in Costpoint. Crosswalk is the translation of an old value to a new value. For example, in a legacy system, there may be an employee ID of 0001 (four digits) The new target database may have a 5-digit ID, so it needs to be padded with one digit – the new value is 10001. Common alpha/numeric values subject to the crosswalk process are employee ID, vendor ID, customer ID, and project ID.
Q. What is key for a successful data conversion?
Christine: Mappings and crosswalks are key for data migration or data restructure because you must have an accurate mapping in order to have good results. Often, the mapping and crosswalk process is underestimated. Testing is also crucial. There can be the false belief that SQL can do anything, the data has been mapped and that tens of thousands of records can be processed in a matter of minutes. However, that may not necessarily be the case because there’s a lot of complexity involved in moving data. The converted data must be validated by testing to ensure that the migration from one system to the next has occurred properly. As a best practice, we normally plan at least one or two test conversions before the full process goes live to ensure data integrity.
A successful data conversion requires a comprehensive data migration and test plan, good data governance, and the appropriate level of expertise. Having an experienced professional with excellent references helps the process go more smoothly.
Q: Don’t tools such as ETL (Extraction, Transformation and Load) make the data conversion process simple?
Christine: Although ETL is powerful and can convert massive volumes of data, there is an old saying “garbage in garbage out.” If the mapping has not been done properly, the power of ETL merely propagates the problem. If data is miss-mapped, the fidelity of the target database is compromised.
Q: What causes mapping or crosswalks to be difficult?
Christine: Data mapping can be a critical challenge during a data migration project. The source and target are unique systems, and each has its own set of data and data logic. You don’t always find the matching relationships. Data mapping is a process where you identify a home location for the source data in the new target system. Any gaps or missing data will complicate data conversion, validation and potentially require many iterations to hone in on the source of any discrepancies. In terms of the crosswalk (an old data value translating to a new value), one to one mapping is the most straightforward however, that’s not always the case. For example, duplicate vendor IDs may accumulate over the years so a consolidation process may be considered. This would be a many-to-one crosswalk where multiple VEND ID must be mapped to one new ID. To complicate matters further, sometimes multiple dimension crosswalks will be required, and this would also present additional challenges. A data migration project usually involves a large amount of data, when a business has just a few dozen records to map. It really shouldn’t be that difficult; but the reality is that when you have a large volume of data to map and when you factor in typos or duplicates, the challenge can grow in complexity.
Q: What are keys to best-practices mapping?
Christine: Understanding both source and target data (what you are converting from and to) is very important. Deploy well-designed and meaningful mapping documents and double checking mapping files. Always verify the work via post conversion validation. It is rare to get it right (or perfect) the first time. It’s inevitable that something was not mapped right, something was missing, and something needs to be changed. So that’s why we first test the conversion and identify issues and modify the mapping and scripts until it’s right. The testing is very important.
Q: Are these mapping processes manual or can they be automated?
Christine: It depends. Sometimes the crosswalk can be automatically generated. The example we used earlier where you modify the existing employee ID and you need to pad it with one character; that can be generated using SQL and it can be automated. If it’s not so clear cut, then it involves manual, human intervention. We often use Excel in crosswalks, especially when it’s a single dimension crosswalk. It can be a very simple 2-column, old value to new value. An important consideration here is having people with knowledge of both the originating legacy system as well as the new one. My expertise is Costpoint, so I understand the Costpoint target data well. But, if I am converting from a homegrown system, I must depend on the client’s IT support team to explain the kind of data they have in their system and where it’s coming from. I know where it goes in Costpoint, so we work as a team to get it accomplished.
Q: Do organizations tend to underestimate the amount of time or resources to perform a data conversion?
Christine: Yes, for various reasons, the required time and effort for a data conversion is often underestimated, which may lead to project delays and budget overruns. A proper discovery process on the front end would help understand the client business objectives, mitigate risks and develop project strategy and a more realistic budget.
Q: What are key issues CEOs or CFOs need to articulate prior to a data conversion project?
Christine: Timeframe is crucial – how much time is allocated to complete the project? Resources is another. Estimating an adequate level of internal resources that are needed for the project to complete successfully is essential. Professional and competent support from the client is critical.
Q: How long have you been involved in data conversion projects?
Christine: I began my ERP consulting career with Deltek and have over 20 years of experience in all phases of Costpoint data conversion and implementation; 8 years as A&F (Accounting and Finance) consultant implementing Costpoint and 15 years as a Principle Data Consultant since 2004 converting from legacy systems to Costpoint including Deltek GCS Premier; restructuring existing data in Costpoint databases, merging multiple ERP systems into a single Costpoint database, splitting a single Costpoint database into multiple databases. I have an in-depth knowledge of the Costpoint suite (both front and back end). I have also successfully completed many GCS to Costpoint conversion projects via the DMX tool over the last 4 years.
About Christine Liu
Christine Liu has over 19 years of experience in all phases of Deltek Costpoint implementation and specializes in data conversion, financial architecture design, configuration, testing/trouble-shooting and end-user training as well as project management. Since 2004, Christine has worked as a Data Consultant, converting data from various legacy systems to Costpoint including Deltek GCS Premier; restructuring existing data in Costpoint databases; merging multiple ERP systems into a single Costpoint database; and splitting a single Costpoint database
into multiple databases. Christine has a unique skill set which encompasses an in-depth knowledge of the Costpoint suite – Accounting, Projects, People and Material Management, government contracting, FAR (Federal Acquisition Regulations), and CAS (Cost Accounting Standards).
Christine has worked with both domestic and international clients in diversified industries including: Aerospace & Defense, Technology, Architecture & Engineering, Communications, Healthcare, Research & Development, Insurance, Manufacturing and non-profit organizations.