Entity Resolution

The same customer can appear in a company's systems many times. The CRM lists Acme Corporation, the billing system lists ACME Corp., the support system lists a contact's email at acme.com and a recently acquired subsidiary lists Acme Holdings Ltd. Unless these records are recognized as the same entity, reports will undercount the customer's value, sales and support teams will lack a full picture and analytics will draw the wrong conclusions. The same problem affects suppliers, products, employees, applications and almost every other kind of business entity. Entity resolution is the work of finding and linking these records.

The problem has a long history. Statisticians developed methods for linking census and health records decades ago, and the model published by Ivan Fellegi and Alan Sunter in 1969 remains a foundation of probabilistic record linkage. Today, entity resolution is used in master data management, customer data platforms, fraud and anti-money-laundering systems, knowledge graphs and data integration of all kinds. Machine learning has improved matching accuracy, and graph-based approaches help resolve entities by looking at their connections as well as their attributes. For data engineers, data stewards, analytics teams, architects and anyone building connected data foundations, entity resolution is an essential and often underestimated step.

Definition of Entity Resolution

Entity resolution, also known as record linkage, deduplication or identity resolution, is the process of comparing records within and across data sources, determining which ones describe the same real-world entity using rules, statistical models or machine learning, and linking or merging them under a single, persistent identifier.

It is a core step in master data management (MDM), is essential for building a reliable knowledge graph and supports a single source of truth (SSOT).

Steps in Entity Resolution

Standardization. Cleaning and normalizing data, such as formatting names, addresses and phone numbers consistently.

Blocking. Grouping records into candidate sets that might match, so the system does not have to compare every record with every other.

Matching. Comparing candidate pairs using rules, similarity scores, statistical models or machine learning to decide whether they refer to the same entity.

Clustering and merging. Grouping matched records into entities and creating a combined or golden record with the best available values.

Review and feedback. Sending uncertain matches to people for review and using their decisions to improve the matching over time.

Matching Approaches

Deterministic matching. Records match when specific fields agree exactly, such as tax ID or email address. It is simple and precise but misses records with errors or variations.

Probabilistic matching. Records are scored on how likely they are to match, based on the agreement of several fields, with thresholds for automatic matches and for review.

Machine learning matching. Models trained on labeled examples learn which combinations of features indicate a match.

Graph-based matching. Shared connections, such as common addresses, devices or contacts, help determine whether records describe the same entity.

Importance of Entity Resolution in Enterprise Operations

It creates a complete view. Linking records gives a full picture of each customer, supplier, product or employee across systems.

It improves data quality. Removing duplicates reduces errors in reports, analytics and operations.

It also underpins connected data. Knowledge graphs, master data and AI systems depend on knowing that two records are the same thing. Without it, AI agents may treat one customer as several or merge several suppliers into one, producing wrong answers and actions.

Entity Resolution and Financial Management

Resolving supplier records shows total spending with each vendor, supporting negotiations and reducing duplicate payments. This also supports third-party risk management, since risk assessments can cover each supplier's full relationship with the company.

Resolving customer records gives accurate revenue and profitability by customer, which supports pricing and account decisions.

Linking applications, contracts and vendors across systems improves visibility of technology spending.

The cost of entity resolution, including tools and stewardship, is usually small compared with the cost of decisions made on duplicated or fragmented data.

Common Challenges

Data quality in source systems is poor, with missing, outdated or inconsistent fields that make matching difficult.

Thresholds are set too loosely, merging different entities, or too strictly, leaving duplicates in place. Testing thresholds on a labeled sample of records helps find the right balance.

Matching is done once as a cleanup project, and duplicates return as new records are created. Within months, the duplicate rate can climb back to where it started.

Privacy and consent rules limit which data can be combined, especially for personal information. Regulations such as the GDPR require a lawful basis for combining personal data, so privacy teams should be involved early.