Advanced Duplicate Check: Architecture, Integration and Production Design (Part 4)

At senior and architect level, Duplicate Check should be understood as an end-to-end design problem rather than a list of isolated features.

Core design map
Why Duplicate Check Matters in MDG Governance: Understand the business and technical purpose of duplicate check in MDG, why duplicates are costly, and where the check fits into the change request lifecycle.

Configuring Search and Match Rules for Duplicate Check: Learn how duplicate check search criteria, matching fields, similarity thresholds, and change request integration are configured for governed entities in MDG.

Tuning and Troubleshooting Duplicate Check Results in Production: Learn how to diagnose, tune, and support duplicate check behavior in a live MDG environment, including handling false positives, false negatives, performance issues, and score threshold adjustments.

Architecture and production criteria

• Change one configuration variable at a time and measure impact before combining changes
• Confirm which change request types and creation entry points (UI, interfaces, mass processing) are in scope for duplicate check early in the project
• Document every tuning change with rationale, validation results, and a rollback plan
• Explicitly verify and document which creation channels (interactive UI, mass processing, interfaces) are covered by duplicate check
• Involve data stewards early to define which attribute combinations genuinely indicate duplicate entities in the business context
• Maintain a regression set of known duplicate and non-duplicate pairs to validate every tuning change
• Monitor search index refresh timing after mass loads and communicate expected delays to stewards
• Position duplicate check as part of the overall data governance narrative when presenting MDG value to business stakeholders
• Route high-confidence duplicate matches to a data steward approval step rather than relying solely on the initial requester's judgement
• Run a baseline data quality assessment on existing master data before enabling duplicate check, so early results are meaningful
• Schedule periodic review of duplicate check configuration as data volume, regions, or business lines expand
• Set realistic expectations with stakeholders that duplicate check reduces but does not eliminate duplicate creation risk
• Track override rates on duplicate warnings to detect when stewards are habitually bypassing valid signals
• Train requesters explicitly on how to interpret and act on potential duplicate results shown during change request creation
• Tune similarity thresholds iteratively using labeled sample data of known duplicates and known distinct records before go-live
• Verify actual configuration scope and capability in the specific target tenant rather than assuming parity across deployment types

Failure analysis and operational risk

• Assuming cloud and on-premise duplicate check configuration options are identical without verifying in the actual tenant
• Assuming duplicate check will automatically block all duplicate creation without human review, leading to disappointment when near-duplicates still get created due to low similarity scores
• Assuming the same search configuration works identically across regions with different address formats, naming conventions, or identifier standards
• Assuming the search index is always current immediately after mass data loads
• Failing to communicate to business users that duplicate check adds a step to the process, causing resistance if not explained during change management
• Failing to document tuning changes, making it impossible to audit why a match rule behaves differently over time
• Forgetting to extend duplicate check coverage to mass creation, interface, or API-driven master data creation paths
• Ignoring data quality of existing master data when enabling duplicate check, so the search runs against a base that already contains duplicates, undermining trust in results
• Ignoring inconsistent population of key matching fields such as tax number or registration ID
• Not aligning duplicate check scope with the specific change request types in use, so the check is skipped for certain creation paths (for example mass creation or interface-driven creation)
• Not distinguishing between warning-level and blocking-level duplicate results when analyzing steward behavior
• Not revisiting or retuning configuration after go-live as data volume and business scope grow
• Selecting search fields based only on technical convenience rather than what data stewards say actually distinguishes duplicate from legitimate records
• Setting a similarity threshold without testing against real sample data, resulting in either alert fatigue or missed duplicates after go-live
• Treating duplicate check as a one-time technical switch rather than a business process requiring requester training on how to interpret potential matches
• Tuning thresholds based on a single complaint without a validation data set, causing regressions elsewhere

A strong production design connects functional or analytical semantics to integration boundaries, security, performance, transport/change control, observability, recovery and ownership. Trade-offs should be justified with evidence such as volume, latency, data quality, user behavior, operational SLA and downstream dependencies. Avoid treating a technically successful configuration or interface as complete until the business result is reconciled end to end.

A consumer goods company implementing MDG for supplier master data found that finance had over 3,000 supplier records, with an estimated 8% being duplicates created by different regional purchasing teams using slightly different company name spellings and address formats. During the MDG rollout, the project team enabled duplicate check on the supplier creation change request type so that any new supplier request first ran a similarity search against existing active suppliers using company name, city, and tax number. Requesters saw a list of potential matches before submission, and were required to either confirm the record as genuinely new or cancel and reference the existing supplier. Within the first quarter after go-live, duplicate creation requests dropped noticeably, and the accounts payable team reported fewer duplicate payment investigations.

During an MDG implementation for a manufacturing group operating in multiple countries, the project team initially configured duplicate check for customer creation using only company name and country. Early testing revealed excessive false positives because many legitimate customers shared generic name patterns (for example common distributor names repeated across regions). The team revised the configuration to weight tax identification number and postal code more heavily alongside name similarity, and introduced a moderate threshold tuned using a sample set of 50 known duplicate pairs and 50 known distinct pairs supplied by the data governance team. After retuning, the false positive rate dropped substantially while known duplicate pairs were still correctly flagged, and the configuration was signed off by the data steward community before go-live.

A shared-services company centralizing vendor master data noticed the duplicate check was blocking a high volume of legitimate new vendor requests because many vendors operated out of the same business park address with shared mailboxes. The data governance team pulled a sample of 40 flagged requests, confirmed 32 were false positives caused by address-heavy matching, and worked with the configuration team to rebalance the match weighting toward tax identification number and normalized legal name. They validated the change against a curated set of known duplicate and non-duplicate pairs before promoting it, and set up a recurring monthly review of duplicate check override statistics to catch any drift.

• Assuming cloud and on-premise duplicate check configuration options are identical without verifying in the actual tenant
• Assuming duplicate check will automatically block all duplicate creation without human review, leading to disappointment when near-duplicates still get created due to low similarity scores
• Assuming the same search configuration works identically across regions with different address formats, naming conventions, or identifier standards
• Assuming the search index is always current immediately after mass data loads
• Failing to communicate to business users that duplicate check adds a step to the process, causing resistance if not explained during change management
• Failing to document tuning changes, making it impossible to audit why a match rule behaves differently over time
• Forgetting to extend duplicate check coverage to mass creation, interface, or API-driven master data creation paths
• Ignoring data quality of existing master data when enabling duplicate check, so the search runs against a base that already contains duplicates, undermining trust in results
• Ignoring inconsistent population of key matching fields such as tax number or registration ID
• Not aligning duplicate check scope with the specific change request types in use, so the check is skipped for certain creation paths (for example mass creation or interface-driven creation)
• Not distinguishing between warning-level and blocking-level duplicate results when analyzing steward behavior
• Not revisiting or retuning configuration after go-live as data volume and business scope grow
• Selecting search fields based only on technical convenience rather than what data stewards say actually distinguishes duplicate from legitimate records
• Setting a similarity threshold without testing against real sample data, resulting in either alert fatigue or missed duplicates after go-live
• Treating duplicate check as a one-time technical switch rather than a business process requiring requester training on how to interpret potential matches
• Tuning thresholds based on a single complaint without a validation data set, causing regressions elsewhere

• Change one configuration variable at a time and measure impact before combining changes
• Confirm which change request types and creation entry points (UI, interfaces, mass processing) are in scope for duplicate check early in the project
• Document every tuning change with rationale, validation results, and a rollback plan
• Explicitly verify and document which creation channels (interactive UI, mass processing, interfaces) are covered by duplicate check
• Involve data stewards early to define which attribute combinations genuinely indicate duplicate entities in the business context
• Maintain a regression set of known duplicate and non-duplicate pairs to validate every tuning change
• Monitor search index refresh timing after mass loads and communicate expected delays to stewards
• Position duplicate check as part of the overall data governance narrative when presenting MDG value to business stakeholders
• Route high-confidence duplicate matches to a data steward approval step rather than relying solely on the initial requester's judgement
• Run a baseline data quality assessment on existing master data before enabling duplicate check, so early results are meaningful
• Schedule periodic review of duplicate check configuration as data volume, regions, or business lines expand
• Set realistic expectations with stakeholders that duplicate check reduces but does not eliminate duplicate creation risk
• Track override rates on duplicate warnings to detect when stewards are habitually bypassing valid signals
• Train requesters explicitly on how to interpret and act on potential duplicate results shown during change request creation
• Tune similarity thresholds iteratively using labeled sample data of known duplicates and known distinct records before go-live
• Verify actual configuration scope and capability in the specific target tenant rather than assuming parity across deployment types

Senior interviews should test whether the candidate can connect the individual lesson areas, diagnose cross-layer failures, explain trade-offs and design a supportable production operating model.

Interviewers commonly ask candidates to explain, in plain business terms, why duplicate check exists and where it sits in the MDG change request process, then probe whether the candidate understands it is a decision-support step rather than a fully automatic block. A strong answer connects the business cost of duplicates (financial, compliance, reporting) to the technical mechanism (similarity search integrated into CR workflow) and shows awareness that results depend heavily on the quality of existing master data and the chosen search criteria.

Interview questions at this level often focus on how a candidate approaches threshold tuning and field selection trade-offs, since this reflects real hands-on configuration experience rather than theoretical knowledge. Strong candidates describe an iterative, data-driven tuning process using known duplicate and non-duplicate samples, explain why identifier fields are usually weighted alongside fuzzy name/address matching, and acknowledge that different creation channels (UI versus mass/interface) may require separate verification of duplicate check coverage.

Interviewers assess whether a candidate can reason about false positives versus false negatives systematically rather than anecdotally, understands that duplicate check quality depends on data field consistency as much as configuration, and knows to validate tuning changes against a regression set. Strong answers reference indexing timing issues as a root cause of missed duplicates and articulate the difference between blocking and warning-level results in a workflow.