Reference Data Management 101: Definition & Walkthrough

Section

Organizations across every industry use reference data to organize and categorize other data. Think of the periodic table of elements, which provides a structure for organizing chemical elements by their name, symbol, atomic structure, and more. This is an example of reference data. 

Source

Or there’s the Universal Product Code (UPC), which is a 12-digit number used to identify and track products as they’re sold, shipped, and ultimately delivered to customers. 

An example of a Universal Product Code used to track a specific item. 

Reference data provides important context to businesses, but when it’s mismanaged it can lead to huge missteps in regards to miscommunication and deteriorating data quality.  

This is where reference data management (RDM) comes in, to ensure all data within an organization is properly classified, defined, and easily accessible.  

What is reference data management?

Reference data management (RDM) is a system for organizing, updating, and consolidating reference data. It can be divided into internal or external data.

• Internal reference data includes company-specific information like product or transaction codes.

• External reference data is provided by organizations like the International Organization for Standardization (ISO). 

Reference data management supports both types of data to ensure your organization runs smoothly and that you’re meeting international compliance requirements. It can also be divided into two varieties:

• Multi-domain reference data: Used across multiple industries, business units, and content types.

• Real-time reference data: Used most often in the capital markets industry or in Internet of Things (IoT) operations that rely heavily on metadata.

Reference data is an extension of master data management (MDM), which unifies organizational data on products, customers, financials, and other assets into a single source. Reference data then categorizes this master data into subgroups for processing, sharing, or streamlining workflows. 

Reference data includes identifiers like:

• Location attributes like country codes, postal codes, and state abbreviations

• Product codes

• Pricing and transaction codes

• Exchange codes

• Languages

• Financial hierarchies

• Currencies

• Measurements

• Industry and classification codes

The importance of reference data management and integration

Data helps organizations evaluate their business operations and performance, iterate on products and ideas, analyze customer engagement, and fuel growth strategies. But it’s impossible to accomplish these tasks if data is unorganized, inaccessible, and out-of-date, which is why reference data management and data integration are essential.  

Challenges of managing reference data

Organizations that don’t invest in managing their reference data face several challenges when it comes to their data governance, like:

• Having to manually maintain tools and update spreadsheets 

• Dealing with duplicate, irrelevant, or incorrect data due to a lack of standardized naming conventions

• Unnecessary time, money, and resources spent on manually  gathering and consolidating data from multiple sources

• Difficulty assigning and standardizing proper values for codes 

• Miscommunication and errors due to a lack of data-sharing mechanisms 

Benefits of reference data management and integration

On the flip side, reference data management comes with a long list of benefits, like improving data usability and quality. More specifically, RDM helps to:

• Improve workflows and increase data accessibility 

• Remove errors and inaccuracies to improve data quality and cleanliness

• Streamline updates for new reference data

• Refine business intelligence (BI) reporting

• Meet regulatory requirements

• Simplify communication between departments

• Reach resolutions faster with accurate data 

Reference data management also helps dismantle data silos and consolidate data into one central system, improving accuracy and increasing security by establishing a single point of access. 

Consequences of forgoing data management

Mismanaged reference data leads to significant consequences that affect business units, customers, employees, and stakeholders. Without a reference data management system, you may see:

• Security and compliance risks like misuse of data, breaches, or privacy violations.

• Poor decision-making based on inaccurate or incomplete data, along with potential lost opportunities.

• Inefficient processes that negatively impact growth because of delays and miscommunication

• Operational issues if data isn’t centralized and standardized across the supply chain

• Customer dissatisfaction when product codes and shipping codes result in incorrect or lost orders 

• Higher costs across the organization, including data management, legal compliance, transportation, and shipping  

Even one error can cause a ripple effect across an organization. Say a business accidentally issues the same product code for two different items, resulting in these products being shipped to the wrong locations. This type of mistake creates a headache for the customer, disrupts your supply chain, and undermines trust that your inventory is correct. 

How reference data management works

RDM can be broken down into three phases: integration, management, and accessibility.  

Integration 

During integration, businesses consolidate data from different sources to create a golden copy by:

• Identifying all data sources.

• Categorizing data into internal and external sets, and removing any overlaps. 

• Analyzing data and implementing rules for maintaining quality, like value length, naming conventions, expiration dates, and so on, according to your business rules. 

• Building efficient extraction processes to consolidate all data sources into one data hub. Ensure future updates and additions follow the same process. 

• Creating hierarchical relationships through data mapping to increase accuracy and accessibility.

Management

Once reference data is integrated into a central repository, managing it becomes the next priority by:

• Preserving data quality by automating quality assurance checks to keep data accurate, updated, consistent, and relevant.

• Auditing data usage and tracking modifications to catch accidental errors or untimely changes.

• Ensure data integrity by setting access permissions and controlling who can edit, change, or delete values.

Depending on your data needs and size, a software platform might be enough to handle routine reference data management. Another option includes building a dedicated data management team consisting of data stewards, analysts, and subject matter experts (SMEs). 

Accessibility 

Accessibility helps you get the most value from your reference data by guaranteeing it's accessible across every team. This is often done by:

• Building delivery infrastructures that allow access to data from requesting applications or individuals.

• Ensuring flexibility for exporting and sharing data across the organization.

• Creating support for future changes either through automation or other extraction processes.

10 best practices for reference data integration

Best practices for data reference management can vary across industries and even across organizations within the same industry. Regardless, to create a solid reference data framework:

1. Sync reference data into one centralized, adaptable system. 

2. Standardize values to ensure data quality and validation. 

3. Implement security measures to protect data and avoid compliance violations.

4. Monitor and audit usage to ensure controlled access to data.

5. Educate employees on proper usage to avoid critical errors or unauthorized access.

6. Follow data governance requirements to protect data and avoid legal repercussions. 

7. Automate processes to ensure updates are timely and accurate.

8. Use external reference data relative to your industry, like SWIFT, ISO, or ICD.

9. Review data sets to remove errors, duplications, or irrelevant data.

10. Work cross-functionally with the data governance team, IT department, stakeholders, and key executives to ensure accountability for their roles in RDM.  

If your business is implementing reference data management for the first time or even updating your processes, create a data repository that includes information about your datasets and values. Doing so helps business users understand how to properly access, interpret, and utilize the datasets. 

Manage your data with Twilio Segment

Twilio Segment’s customer data platform (CDP) helps collect, clean, and consolidate customer data across an organization. With Protocols, businesses are able to enforce a universal data dictionary to standardize naming conventions and avoid inaccuracies in their data collection (e.g., duplicate events or misnamed entries). Protocols also automates the QA process and is able to block bad data before it reaches its downstream destinations and skews decision making.