6 questions to ask before any big data migration

To prepare for a recent presentation I gave on big data migration at ERworld 2015, I went back and examined lessons learned from government data management projects I’ve consulted, and how they suggest about  the transformational mindset that big data demands. There are many takeaways, but here are the highlights: six questions that can help any agency executive better approach and execute a major transformational initiative. 

#1 – Why do you need to migrate the data?

Is the “why” the same for your team, colleagues, and others involved? What is your purpose? IT’s goal might be to cut down on costs, while business users need new data sources to better serve the mission or obtain a competitive advantage.

It is important to define what big data means to you, to users within your organization, and to external users. Does everyone share the same definition?

Big data means different things to different roles in the organization, so ask around. What does it really mean to them? Not just the definition in the dictionary, but what does big data truly mean to the end user? To the organization? To the compliance officer?

Also, do you have a data governance practice in place -- that you actually use? Who is responsible for its oversight? Make sure to get him or her involved right away. An effective data governance framework can help organizations with consistent definitions, manage better by measuring and tracking the exchange of data; leading to enhanced decision making, communication, and collaboration while providing disciplined, controlled framework and processes.

#2 – Who is involved?

Don’t just mentally prepare yourself -- you also must ready your team and others who will be affected. Who are the key players in this project? What about the stakeholders, and the subject matter experts? Identify them, communicate with them, and engage them. If someone knows and understands the business data at hand, they should be included in some way. Create a communication matrix and plan.

#3 - What is the data used for?

Gain insight by assessing and documenting your current state. Do you honestly know what the data in question is being used for? Do you know what decisions are made as a result of it? Sometimes you may discover that data is disappearing into a vacuum and is not utilized at all, or is used only within a specific functional stovepipe for a specific functional purpose. Is data turned into metrics to measure activity for resourcing? Is it turned into metrics to measure the business? Are they ever confused? To optimize the migration process, don’t just check the box. Investigate and then design the initiative.

By taking the time to operate in detective mode, you will increase your likelihood of success exponentially.

It is the IT department’s responsibility to manage big data -- but for whom and why? Step outside your current perspective to discover the business reasons and importance for the uses of this big data. Is the organizational communication matrix and plan documented in a way that addresses who has access to what data, when that data is accessed, and at what point in the process? What data is needed? What data is shared by multiple departments within the organization? What are the desired outputs?

#4 – When will you migrate?

It is very important for all parties to be on the same page regarding timelines and milestones. For example, when answering the question of "when," the response can vary widely. I see plenty of people stuck at this stage because they are unable to agree on what needs to be migrated when. There may very well be interdependent requirements that must be accomplished before the next activity can take place. Managing timelines and inputs is critical to staying on time and within budget.

#5 – Where should the data go?

The “where” is a crucial question. Having formal internal documentation and an agreement of expectations in place enables easier detection of potential impacts and allows for reasonable risk management as change arises. It also allows for a set change management plan to reference as the “where” changes from the “as-is” location to the “will-be.”

#6 – How does data get migrated?

Now you must tackle “how” you will get from Point A to Point B. What actions should your organization take? What are the mechanisms, tools and technology that will help you achieve your data migration goals? What conditions need to be created? What changes to existing processes, training, source documentation, etc., need to be made?

When it comes to transforming your organization through big data migration, there often is pressure to jump in immediately. Don't. When mentally preparing to undertake such an initiative, you must fully evaluate and understand the lifecycle of the data. Ask these six questions (while relying on a formal data governance framework) to understand your projects’ impacts to people, processes and technology.

You may discover new sources of data that you didn’t realize should be part of the project scope. Or you may find that the planned migration does not make business sense. That’s when relying on a strong framework, in the form of documented approaches and processes is truly empowering. Agencies can proactively transform business environments by asking the right questions up front, and then using those answers to create a strategic plan.

Many organizations are shocked to realize that the key to easier migration of big data is so simple -- proper planning. By finding the balance between the preparation and execution stages, you can truly “move mountains” of data in a way that will significantly and positively impact your agency mission.