Data migrations are time-consuming, painstaking, complex and even potentially harmful if you don’t follow the proper precautions. Importing a couple hundred records into the Salesforce® platform? Easy enough. But what if you need to transfer hundreds of megabytes of data from another system, across multiple objects? That project can throw roadblock after roadblock at you if not approached in the right way.
To learn the proper Salesforce data migration best practices, I interviewed our in-house guru, Christian Aguirre. He has over 7 years of experience in computer science and information technology, and he agreed to share his path to technology and his advice on the migration process with me.
Below is a transcript of our interview about his story and data migration tips, which was condensed and edited for clarity and formatting.
Kevin Binder: So, before we begin, would you mind telling me a bit about yourself? How did you come to Torrent Consulting?
Christian Aguirre: Sure. So, I was born and raised in Guatemala City, Guatemala. My dad is an electrical mechanic, and my mom is a hairstylist. I’d describe my parents as really hard-working people. Even though they didn’t have college degrees, they always pushed me to pursue higher education and to fight for my goals.
Growing up, we actually lived in one of the city’s “red zones,” where there were high crime rates and a lot of gang activity. But then, because of the dedication I had for my education — specifically learning English — I earned a scholarship to go to college in the United States and study computer science. When I came back from my freshman year, I pushed my family to move to a more peaceful environment.
Once I graduated, I started working as a software developer using Microsoft® technologies. After a few years of that, I joined Torrent Consulting and familiarized myself with the Salesforce platform.
KB: Ok, wow. And so how did you end up pursuing technology? Why are you passionate about it?
CA: So, it’s a combination of my passion for mathematics and my belief that technology fosters equality.
When I was a teenager, I was passionate about mathematics and its applications. So, I actually wanted to be a math major in college. But then I realized that, in a country like Guatemala, it would be difficult to get a job as a math major. So I started studying computer science because it’s an applied area of mathematics.
But besides all of that, I strongly believe that technology expertise can bring opportunities to all people, no matter their nationality, race, ethnicity, gender or socioeconomic status. If you work hard enough to increase your knowledge of technology, limitless opportunities will open up for you.
KB: So I almost don’t want to, because this story is so incredible, but I unfortunately have to turn the conversation to data migration now.
As you know, there are so many options for importing and exporting data to and from the Salesforce platform. Like Data Loader, Dataloader.io and Workbench, to name a few. Which tools do you prefer to use?
CA: So, I personally prefer Data Loader because it’s easily accessible from my local computer — it’s already installed there. In my opinion, it’s very fast and efficient. Plus, the program interface is very intuitive. There’s just one tab for settings, and that’s it. And all of the operations you need — insert, update, upsert, delete, hard delete, export and export all — are defined in different buttons.
It’s a tool that’s well-documented by Salesforce. In general, you want to avoid using solutions that don’t have enough documentation, if possible.
But also I think it’s important that it’s available at no additional charge. There are no limitations saying that you can only upload a certain number of records with a certain subscription. You can upload as many as you want, essentially, so long as your Salesforce storage allows you to, right?
KB: Makes sense. Okay, let’s say you’re starting a data migration. What are your recommended steps for the process?
CA: (Laughs) Man, that’s the biggest question on your list. This is going to take a while (laughs).
KB: (Laughs) That’s fair. So, feel free to go quickly through the steps and then we can dive more deeply into the important ones after that.
CA: Okay. So here’s the process that I’ve found that’s best to follow. And this is backed up by Salesforce’s documentation and our internal knowledge:
1. The first thing is to engage in a discovery of the data and migration requirements involved. That involves a whole bunch of questions that I’ll get into later.
2. Then, the second step would be to create templates for the data and provide those to whoever has the necessary data so that they can put it in a specific format. You’ll want to highlight the required fields and define the format of any fields that need a particular format.
3. From there, you’ll get data imports from the teams with the data. Make sure the files are de-duplicated. And clean them as much as possible.
4. Another step that I recommend, if there’s already data in Salesforce and you’re updating or deleting records, is to back up that data before you run any loads. In case the updates need to be rolled back, right? But that’s only if we’re updating data. If we’re just importing data, it doesn’t really apply.
5. After that, the next step would be to identify which validation rules, automations and triggers might fire for the Salesforce objects involved when we run the import or update. Based on that, we need to decide which ones should be turned off before we run the import process.
6. Then, we’d do a test run in a sandbox of a small set of data. Troubleshoot any import or update errors and then do a quality check on the data we loaded.
7. After we’re done with the sandbox test run, we can import a small set of data to production and review it, just to make sure everything went well and everything has been mapped as expected.
8. And once the small set of data that was imported has been verified, then we can import the rest of the data to production.
KB: Yeah, that’s an incredibly thorough list and gives you everything you need to run through. Very helpful, so thank you.
Now, to dive into the individual pieces, you called out “discovery” as the first step. What’s involved there? What should an admin be looking at?
CA: When you’re doing a discovery of the data and migration requirements, there are a lot of questions you need to ask. Here are some good ones to start with:
- What are the objects involved in the data migration?
- Of all the objects involved, how are they related?
- Are there any new objects we need to create in Salesforce to hold certain data we’re migrating?
- Which fields do we need to map from our data source?
- Which fields are required in Salesforce?
- What are the unique identifiers for each record? Do we have any?
- Do we need to perform any data transformation work as a result of the answers from above?
So, that’s the first step (laughs).
KB: (Laughs) So this is quite the process. Now, you just talked about data transformation and cleansing within the migration process. What’s involved there?
CA: Well, as I said, you want to make sure your data is deduplicated and put into a format that Salesforce will accept. For example, certain field types like phone numbers, dates and addresses need to be formatted a certain way before they’re imported into Salesforce. Also, if you’re loading data into a Salesforce picklist from ExcelTM, you’ll need to make sure your data matches those picklist values before you run your loads.
KB: I know the data transformation step typically takes the most time in the process. Do you have any tips on how to save time on that piece?
CA: Yeah, when we talk about cleaning data, Excel is actually a great tool. So long as you master functions like “Sort,” “Filter,” “VLookup,” “Format Cells” and “Find and Replace,” you shouldn’t have any issues cleaning data using Excel. I recommend that any admin master those if you need to clean up data.
Making sure your data is in the right format in Excel before attempting an import to Salesforce is the easiest way to save time during this process.
KB: One of my questions was going to be about the precautions admins should take to prevent irreversible mistakes, and I heard you mention backing up your data as a crucial step. Is that your recommended approach to avoiding messing up data in your production org?
CA: Backing up your data definitely helps. Otherwise, you can’t rollback any mistakes. But that’s also one reason I recommend doing a test run in your sandbox before importing to production. It helps you catch any problems before you make any changes to production data.
KB: Ok, so next take me through the step of turning off automations, validations and Apex triggers. Why is that important?
CA: Well, depending on what those validations and automations do, we might need to temporarily deactivate them to avoid accidentally sending undesired emails or notifications, or kicking off a process when we shouldn’t. Additionally, leaving these rules on could cause records to fail during an import or update. After the migration process is complete, it’s important to remember to reactivate all the automations and validation rules that we turned off.
KB: Makes sense. And that takes us to our last deep dive, about validating a test import and running quality control on it. What’s your recommended approach there?
CA: So that’s really two separate parts. First, you want to troubleshoot any import or update issues by using the error logs generated by each import process. It doesn’t matter which tool you’re using — Data Loader, Dataloader.io or Workbench — all these tools tell you which records failed and why, right? So you need to troubleshoot using those logs and fix the errors in your data before trying again.
And even once you’ve successfully loaded your test data, you need to review that load by checking the count and quality of that data. You need to do that in Salesforce, either by building reports or executing SOQL queries.
KB: I think we all know that data errors can be the most frustrating part of any migration. Does the test import and quality control take care of those issues, or is there anything else you recommend?
CA: That does help, but I actually think the step of turning off validations, automations and triggers is more important when it comes to avoiding errors. And remember that, even if you’ve turned them off in your sandbox for your test load, you’ll also need to turn them off in production for the real thing!
KB: Awesome, that’s a fantastic overview of everything you need during the process. Are there any important steps admins often forget?
CA: Yes, a lot, actually! Most commonly, people forget the step I just talked about: turning off unwanted validations, automations and triggers. They also might skip the test imports in sandbox or production, which is risky. Finally, some admins miss parts of the discovery process. So, they might not have a clear understanding of how the objects involved are related before starting the data migration.
KB: Speaking of relationships, let’s say you’re migrating multiple objects into the Salesforce platform. Is there a particular order you should load different objects in?
CA: Yes, your object relationships, in general, dictate the order of data migration. You want to load in the “highest level” object first and then work your way down.
Here’s an example: Let’s say you’re migrating Accounts, Contacts, Cases and Events to Salesforce. Accounts are the highest level object, each Contact looks up to an Account, while Cases can look up to Accounts and Contacts. Then, Events can look up to Accounts, Contacts and Cases. So, you should load your data in that order: Accounts, then Contacts, then Cases, then Events.
KB: Well, that does it for my list of questions. Any other best practices you want to share that I haven’t asked about?
CA: Yeah, if you’re importing Salesforce data from another system with its own unique IDs, I recommend creating custom Salesforce fields for each object to store those external IDs. Then, you can use those external system IDs in later data loads to find those records in Salesforce, to update them or relate them to other records.
Have unanswered questions about Salesforce data migration best practices? Send us a message to get in touch and see how our experts can help you through the process!