Admin Best Practices: How to Eliminate Salesforce Duplicates with Delpha
Published by Chethan Kumar on November 29, 2022
Yup, we are deduping data in this article; unfortunately, not cocktails. Well, if you follow my suggestions here then you could be spending less time deduping data and more time deduping–aka drinking–your favorite beverage 😉
Duplicate data prevents a true 360-view of customers and harms business data quality and operational success. It impacts the entire org, including admins, decision-makers, sales reps, marketers & customer support professionals.
The headache of deduping Salesforce data is not new, but the scale of the problem has increased significantly with the evolution of our tech stacks and escalating data decay rates (exacerbated by the Great Resignation and current tech layoffs). These factors make eliminating duplicate data more challenging but not impossible. Salesforce does offer native duplicate management features; however, these features are limited and unable to merge duplicates automatically.
In this article, I’m going to showcase a new solution on the AppExchange called Delpha, a DataOps platform that detects, monitors and merges data in standard & custom objects within Salesforce. But specifically, I’m going to focus on the features relevant to Salesforce Admins and Data Stewards which they can use to optimize the data quality for their org using Delpha. I will also share some best practices that answer many of the common questions I get from our current clients.
1. What is Delpha and how is it different from existing solutions?
Delpha is an AI-powered DataOps Platform that is able to assess your entire org and score the potential duplicate matches. We offer clients the ability to automate the merge with specific rules to fit your business or process the duplicates in large batches or individually by your end users. We eliminate the restriction of how many duplicates can be merged at once and save admins significant time in merging duplicates.
Our unique approach ensures clients can have clean Object records in real-time and take a proactive approach to eliminating any newly created duplicates ensuring everyone involved in the sales cycle has a true 360-view of their leads, contacts and accounts. Our solution is native to Salesforce and executes inside your org so no need to import or export data unless you want to import older, legacy data sets into your org to clean and merge altogether. To learn more or see how it works, you can watch our recorded demo here: Delpha Deduplication Demo.
I will now delve into the specifics of our solution and share insights on how to optimize the solution to eliminate all your duplicates as well as share answers to some common questions.
2. What is the Delpha Data Steward View and how can it help with my deduplication strategy?
Our Data Steward View is a listview that is populated with all the detected duplicate record pairs from our AI model and conveniently displays the field values that were used, the confidence score of the pairing and the status. The columns or fields can be customized and rearranged as per your preferences.
The Score provides the matching confidence between two records. The Status, Processed & Comment fields together indicate the current status of the duplicate pair. The Processed field is true if the duplicate pairs are successfully merged or if the merge fails due to any error. The Comment field provides additional detail about the merge & captures any issues detected during the merge. All of these fields are automatically populated by Delpha.
The value of this Data Steward View is it allows human oversight over the deduplication process and gives admins the power to process large batches of detected duplicates instantly. Some organizations may want that oversight ability or to have a little more control over the merge processing rather than letting them be merged automatically.
What I love about this list view is it provides flexibility in what’s displayed and can be easily filtered on your desired criteria. For example, clients can choose to filter the data on certain score levels and process all the data in two clicks. Once filtered, say anything above 95, all it takes is one click to select all and one click to change the Status to get merged. The size of the batch doesn’t matter; merge as many as you’d like!
3. How does the scoring work, and can the client modify the model?
All the duplicate pairs identified have a matching confidence score which is the similarity score between two records. Delpha uses AI & ML algorithms on the defined fields to identify the similarity between records. It tells you how confident Delpha is that the records are duplicates. The higher the score, the higher the similarity.
And yes, clients have full power to change what fields they want to send to the model and with the preferred weight for each field.
The fields can be defined or modified on DDQ Duplicate Fields.
The field type can be:
- Boolean: Exact matching, the score can be zero or the weight assigned.
- Fuzzy: All fields, excluding Boolean, use fuzzy matching. The score can range between 0 & the max value.
The confidence score on the duplicate pairs can be improved using the following techniques:
- Increasing Weight on Key fields
If all the fields have equal weightage then each field equally contributes to the final score. Increasing higher weightage on key fields results in a higher score whenever the field matches.
- Using the Mandatory field for scoring
The mandatory field is a checkbox used for rectifying scores because of empty fields. As the name itself indicates, the field values are compulsory for scoring. If the field values are empty, the score goes down.
- Defining expression
In many cases, we use multiple fields for identifying duplicates. Yet, we can be sure that if some of these fields have an exact match, then they are definitely duplicates. For example, if I use five fields for identifying duplicates, but I want the scoring to reflect with certainty if certain fields such as Name & Email or Name and Phone have an exact match then I can achieve this using expressions.
4. How do I auto-merge duplicates?
Auto merge is a feature in Delpha which automatically merges all the records above a particular confidence score. All the duplicate pairs identified have a matching confidence score. This matching score is used for automation. To see the list of duplicates on the data steward view, navigate to the Duplicate Records tab on Delpha Deduplication App.
As you can see, the record and the matching record field values are displayed side by side, along with the matching score. To automate the merge, navigate to Duplicate Objects tab on the Delpha Deduplication App.
In the above view, the AI threshold corresponds to the score for detecting duplicates, while the Auto-Merge Threshold corresponds to the score equal & above. It’s the Auto-Merge Threshold number which will trigger records to be automatically merged using a batch process at the end of the day. Modify the score corresponding to the object to change auto merge threshold scores.
5. How to mass merge duplicates using Delpha?
Auto merge of duplicates is used for duplicate pairs with a very high matching score, but records below the auto-merge threshold will need human decision either in a batch or individual processing. This can be achieved from the data steward view.
The Delpha Data Steward View allows you to mass merge up to 200 record pairs (Salesforce list view limitation) for each batch, but you can process unlimited batches with Delpha. To see the list of duplicates on the data steward view, navigate to the Duplicate Records tab on Delpha Deduplication App.
You can compare the record & matched record field values side-by-side. After comparison, select the list of duplicate pairs to be merged and update the status field to Auto Yes. The batch process running in the background will merge the duplicate pairs at the end of the day.
6. How to increase the frequency of Mass Merges?
Currently, the auto-merge & mass merge of records occurs once a day at the end of the day. There might be some scenarios demanding the need to increase the frequency of the merge. To increase the frequency of mass merge, follow these steps:
- From Setup, search for Apex Classes in the Quick Find search bar.
- Click on Schedule Apex & select DDQ_DuplicateRecord_AutoMerge Apex class.
- Schedule it to run daily.
- Create multiple schedules depending on the desired frequency.
7. Why did the merge fail & where can I get the logs?
There might be scenarios where a merge fails either due to permissions, validations, triggers or flow dependencies that prevent the records from being merged. You can track merge failures on the Delpha Deduplication dashboard. On the dashboard, click on the Duplicates Failed to Merge tile to get additional info.
To see the merge failures on the data steward view or DDQ Duplicate records list view, You can create a list view on DDQ Duplicate Records with the desired fields along with a filter on the comment field as shown below:
8. How can I mark potential duplicates as not duplicates?
The status field & processed on the data steward view can be used to mark potential duplicate pairs as not duplicates. From the data steward view select all the pairs which need to be ignored. Update the status field to manual No & Processed value as true. This prevents duplicate suggestions on the Delpha assistant and prevents merging.
9. Can users tell Delpha not to analyze certain records, or can Delpha analyze a subset of records?
Yes, Delpha allows you to ignore records using the DDQ Do Not Compare field. Do Not Compare is an editable formula field at the object level. If the field value is true, then the record is ignored from the duplicate comparison. To ignore records on Accounts, follow these steps:
- From Setup, select Object Manager.
- Select the Object where you want to ignore records from being compared.
- Search & edit the field DDQ Do Not Compare.
For example: If you want to ignore all the accounts which are of the type Partner & Partner- Prospect you can modify the formula as shown below.
10. Can Delpha automatically choose golden records as the Master Record?
Golden records are the single source of truth created internally or the records with reliable information obtained from an external source. It might be necessary to preserve these records & make them the master record by default. This can be achieved by Delpha using the DDQ Is Golden field. DDQ Is Golden is an editable formula field at the object level. If the field value is true, then the record is selected as the default master both during Auto Merge and on the duplicate conversation that pops up when on a specific contact record. To modify these fields, follow these steps:
- From Setup, select Object Manager.
- Select the Object where you want to set the golden rule
- Search & edit the field DDQ Is Golden.
For Example: If you want to make all the accounts that are from the primary database as the golden record( Master by default). You can modify the formula as shown below.
I hope this article helped you learn more about how to get the most productivity and results from using Delpha as your deduplication solution. Feel free to reach out to me if you have any other questions or leave them in the comment section below.
Book a demo
