Does your organization have a strategy for data quality control? Improving data quality and reducing operational costs requires a solid plan. Establishing a strategy to manage data quality and set organizational standards is essential.

To develop and execute a data quality strategy, you need both a dedicated team and a well-defined process. Here’s how to structure your team and develop effective processes.

Building Your Team

Start by recruiting a data governance team. This team will set clear data definitions, create comprehensive policies, and oversee the documentation process. They ensure that data is collected, managed, and integrated properly across the organization.

Your team should include experts from various functions within the organization. A cross-functional team fosters a data-driven culture and creates data champions throughout the company.

At the helm of this team is typically the Chief Data Officer, who manages the team’s focus, communicates procedures, and monitors success. This leader forms an executive committee with leaders from different departments like finance and marketing. The committee is responsible for developing and overseeing data governance policies and processes.

Mid-level managers from different departments join the team to champion the data governance strategy and ensure collaboration across functions. They define processes, establish data quality metrics, and promote best practices.

Finally, the team assigns data owners, stewards, and users. Data owners manage compliance, administration, and access control. Data stewards act as intermediaries, interpreting data and creating reports. Users are responsible for entering and utilizing data daily and reporting any irregularities.

Define Scope

When implementing a data quality strategy, start with business processes that can benefit immediately from improved data quality. Choose projects with clear, identifiable issues. Begin with smaller projects for quick results, which will help garner executive support for larger initiatives. Each project should have a clear cost estimate and timeline.

Map Data to Key Business Processes

Once the initial project’s scope is defined, map the data flow within the process. Understanding how data moves through your organization and which business processes it impacts is crucial. This mapping helps you see the bigger picture and identify areas for improvement.

Analyze Financial Implications

After mapping the data flow, analyze the financial implications. Poor data quality might affect more areas than initially thought, revealing greater cost-saving opportunities. Collaborate with business management, accounting, and finance to ensure accuracy and gain support for future projects.

Select the Right Technology

Determine the technology needed for data quality evaluation. A diagnostics tool for data discovery and profiling is essential. This tool helps evaluate data set differences over time, quantify outcomes from cleansing, and estimate the project’s ROI.

Determine Data Quality Metrics

Choose metrics to capture the business impact of data quality initiatives. Metrics can range from simple to complex, depending on the data elements involved. Key indicators might include relevance, completeness, timeliness, accuracy, and consistency. Link these metrics to business initiatives to communicate the value of data quality projects.

Establish Best Practices

As you learn from each data quality project, establish best practices. Consistently using these practices will highlight the importance of data quality and influence a cultural shift within the organization. By demonstrating measurable results, you can advocate for the ongoing importance of data quality.

By following these steps, you can develop a robust strategy for managing data quality that supports your organization’s long-term success.

With executive confidence dwindling and the high costs associated with poor data, it’s crucial to ask: How bad is your data quality? More importantly, how can it be quantified? US businesses lose an estimated $611 billion annually due to data quality problems, and less than 33% of companies trust their data’s quality.

Understanding your data’s quality is essential. Check out our previous post on the cost of bad data for more insights. This post will guide you through a quick method to measure your data quality using a simple yet effective approach.

We recommend the Friday Afternoon Measurement (FAM) method to assess data quality (DQ). This method provides a clear, actionable score for your data quality. According to the Harvard Business Review, 47% of newly created data records contain at least one critical error, and only 3% of data quality scores were rated as “acceptable,” even by the loosest standards. These poor scores span across all business sectors, both private and public.

How to Use the FAM Method

Here’s how you can apply the FAM method in four straightforward steps to get a DQ score.¹

Step 1: Gather the last 100 data records your team used, such as setting up a customer account or delivering a product.

Step 2: Invite two or three colleagues who understand the data for a two-hour meeting.

Step 3: Review each record with your colleagues, marking obvious errors. This process should be quick, usually taking no more than 30 seconds per record. In some cases, you may need to discuss whether an item is incorrect, but typically, errors like misspelled customer names or misplaced information will be immediately apparent.

Step 4: Summarize the results in a spreadsheet. Add a “record perfect” column, marking “yes” if there are no errors and “no” if there are any.

To interpret the data, extrapolate the errors. For example, if only 40 out of 100 records are error-free, you have a 40% DQ score and a 60% error rate. This error rate can be quantified using the rule of 10, which states it costs ten times more to complete a task with defective data than with perfect data.

For instance, if your team must complete 100 units per day at a cost of $1.00 per unit with perfect data, the daily cost is $100. However, with only 40% perfect data, the total cost would be:

$\text{Total cost} = (40 \times \$1.00) + (60 \times \$1.00 \times 10) = \$40 + \$600 = \$640$

As shown, the cost increases over six times when the DQ score is not 100%. Reducing errors by 50% in this scenario would result in a 42% reduction in daily costs. Imagine the savings your organization could achieve by improving data quality.

By following these steps, you can gain a clearer understanding of your data quality and take actionable steps to improve it, saving time and resources for your organization.

¹ Thomas Redman, Harvard Business Review, Assess Whether You Have a Data Quality Problem

Executives often harbor skepticism toward organizational data. Understanding the financial impact of bad data is a crucial first step in earning their trust.

 

Why Executives Distrust Their Data

The value of enterprise data is determined by a variety of factors, including accuracy, clarity, and community input. Any deficiencies in these areas can turn valuable data into a liability. Inaccurate data can distort summaries or bias models, leading to poor decisions, missed opportunities, damaged reputations, customer dissatisfaction, and increased risks and expenses.

Such errors can have a significant impact on business decisions and, ultimately, the bottom line. As data volumes and sources grow, managing quality becomes increasingly vital. Unfortunately, data errors are common, leading to widespread mistrust. According to a Harvard Business Review study, only 16% of managers fully trust their data.

A study by New Vantage Partners highlights more reasons for executive concern, especially among those leading data-driven transformations. It identifies cultural resistance, lack of organizational alignment, and agility as major barriers to adopting new data management technologies. Notably, 95% of surveyed executives cited cultural challenges, stemming from people and processes, as the main hurdles. There is a clear need for tools that can be easily adopted to improve data management processes.

 

The Cost of Poor Data

Despite low trust in data quality, executives acknowledge its importance. Organizations are beginning to understand the high costs associated with poor data quality. Experian Plc. found that bad data costs companies 23% of revenue globally. IBM estimates the total cost of poor data quality to the U.S. economy at $3.1 trillion per year.

These costs primarily arise from initial errors that trigger costly reactionary responses. According to 451 Research, 44.5% of respondents manage data quality by identifying errors through reports and then taking corrective action. Another 37.5% rely on manual data cleansing processes.

Highly skilled data analysts spend valuable time manually fixing errors. Syncsort reports that 38% of data-driven analysts spend over 30% of their time on data remediation. Similarly, an MIT study found that knowledge workers waste up to 50% of their time on mundane quality issues, and for data scientists, this figure can reach 80%. This time could be better spent uncovering insights, solving complex business challenges, or generating revenue.

Is your data a liability? Data should support knowledge workers and empower executives to make the best decisions. In this post, we will cover the key characteristics you can use to decide if your data is an asset or liability.

 

Data Asset versus Liability: The Key Indicators of Data Value

The value of a data set goes beyond just the information it holds. It’s determined by the data’s ability to address a specific need. Data quality is the foundation of its usefulness. Below are the characteristics that define data value. A deficiency in any of these can render the data useless, sometimes leading to unknown expenses and problems. Understanding these indicators will help you assess whether your data is an asset, a liability, or a risk.

1. Relevance: Data must be relevant to the needs of the data consumer. For instance, a custom tailor expanding into shoes may not know customers’ shoe sizes but can use height data, which correlates with shoe size, to make inventory decisions.
2. Completeness: Completeness means having all necessary details. Missing columns in a table, incomplete rows of data, or a cut-short video are examples of incomplete data.
3. Timeliness: Timeliness refers to providing data by the required time. While some systems offer real-time results, others operate on batch processes that can delay critical information.
4. Accuracy: Accurate data is crucial. Inaccuracies can spread across systems and models, leading to misguided decisions. Unknown inaccuracies can make data seem like an asset when it’s actually a liability.
5. Precision: More precise data can be used in a wider variety of applications. Measurements in inches are more precise than those in feet, and high-resolution images are better for zooming and large-format printing.
6. Consistency: Consistency means maintaining the same data types and precision across all fields. Changes in data types, distribution formats, or computation formulas can lead to additional costs for the consumer.
7. Uniqueness: Data should be free of duplicate information. Multiple entries for the same person, for example, can be confusing and costly to resolve.
8. Accessibility: Users should be able to easily discover and access the data. Features like semantic search, workflow authorization, and easy-to-use interfaces enhance accessibility.
9. Understandability: Data should come with metadata, detailed documentation, and lineage to help users understand it. Incorrect documentation can be as harmful as inaccurate data.
10. Interoperability: The format of data distribution affects how easily it can be used with different technologies. Using industry-standard formats ensures high interoperability.
11. Community: An active community of contributors and users enhances the value of data. Collaboration and feedback make data more reliable, comprehensive, and understood.

A synergistic relationship exists between accessibility, understandability, interoperability, and community. Modern data catalogs that utilize knowledge graphs and semantic search, like data.world, leverage this relationship to empower data-driven organizations. Knowledge workers can find, understand, use, and share data assets effectively.

By understanding and evaluating these key indicators, you can better determine whether your data is a valuable asset or a costly liability.