At MPP Insights, we work with data every day, and if there's one thing we've learned, it's this: if the data going into a system is bad, the results coming out of it will be bad too, no matter how good the system itself is.
Wrong reports, bad AI predictions, broken dashboards, all of it traces back to this one idea. In the data world, we call this "garbage in, garbage out."
This is why data quality checks exist. They make sure the data used in reports, dashboards, and AI models is reliable.
We'll also cover what data quality means, the most common problems in real datasets, and an implementation example using our dbt data quality library.
What Is Data Quality?
Data quality is a measure of how usable your data is for decision-making. It is important to use "good data" for making reports and dashboards, and "good" doesn't simply mean "correct data."
Data can look fine at first glance and still be useless in practice. Good data has to be accurate, complete, consistent, valid and timely so it can be used for analytics.
Data quality isn't just checked once, it can be checked at several different points as data moves through a business.
- Data can be checked right when data is first collected and stored.
- It can be checked again as that data moves through pipelines from one system to another.
- It can be checked once more when data lands in a data warehouse or lakehouse, the larger systems where data is stored for analysis.
Where and how data gets checked depends on the situation, the tools in use, and what the business actually needs.
Common Data Quality Issues
These problems show up in almost any dataset, and without catching them, the reports and decisions built on that data become unreliable.
#1: Inconsistent values
The same thing gets written in different ways across three different tables. The system treats each version as a separate value.
To show what this looks like in practice, here's a case study from our own work, where we matched hand-typed Armenian addresses to an official registry with 90% accuracy.
Case Study: Armenian Addresses Can Be Written 4 Ways. How We Matched Them to Official Lists With AI
In Armenia, different people write the same address in different orders.
The Challenge
One of our clients had two sets of addresses in their data. One set was typed in by hand, the other came from an official reference list of Armenian cities, villages, and streets. The task was to check whether the hand-typed addresses matched the reference list, even when people made typos or spelled things differently.
This is an inconsistent values problem: the same place gets written in different ways, and the system has no way to know they mean the same thing. You can't list every possible misspelling in advance. Instead, you need a system that can look at two entries and judge whether they probably mean the same place, even when they're not written the exact same way.
Our Solution
We built AI-powered pipelines to handle this. The pipelines used directories of Armenian cities, villages, and streets to clean up the hand-typed addresses first. Then they compared the cleaned addresses against the reference list. This approach reached 90% accuracy.
#2: Duplicate records
A customer signs up through your website, then signs up again later through your mobile app, using a slightly different spelling of their name.
Impact: Your system now has two records for one person,which makes your customer count wrong and can lead to sending the same email or offer twice.
#3: Missing values
A new order comes in, but the shipping address field is left empty.
Impact: The order can still appear in general reports, but it cannot be processed for delivery or analyzed reliably by location until the address is completed or otherwise handled.
#4: Invalid values
A discount is entered as 150% where the allowed range is 0–100%, or an order quantity is entered as -5 in a process that only accepts positive quantities.
Impact: Values that violate defined business rules can cause incorrect calculations or unexpected processing unless they are rejected or handled explicitly.
#5: Different date formats
A transaction date gets logged as 07/06/2026 in one system, and 2026-07-06 in another. Different systems can read the same date differently, some read it as July 6, others as June 7.
Impact: That mismatch can make your timeline wrong, especially when comparing data across systems.
#6: Outdated data
A product listing still shows last year's price, even though it changed three months ago.
Impact: You end up making decisions based on a version of reality that no longer exists.
#7: Data type mismatches
One system stores a customer ID as text, "1042," while another stores it as a number, 1042. These look identical, but they're stored differently.
Impact: Systems can fail to match or compare them correctly, which can cause data to go missing from reports without any obvious error.
#8: Broken relationships (Record linkage)
A customer's account gets deleted, but their past orders still reference that customer's ID. When one piece of data depends on another, but the connection breaks, the record becomes orphaned.
Impact: Those orders now have no clear owner, which makes them hard to trust or use.
Other Data Quality Problems AI Can Help Address
In our case study, we used AI to match addresses that were written differently but meant the same place. Similarity-based and AI-assisted tools can support other data quality tasks as well:
- Duplicate records: AI tools can compare records and flag potential duplicates for review, even when the fields do not match exactly. For example, two records may represent the same customer but use a nickname in one record and a full name in the other.
- Record linkage: AI tools can suggest likely matches across tables when linking fields are inconsistent or incomplete. These suggestions should be validated before relationships are updated.
- Missing values: AI tools can flag missing fields and, where appropriate, suggest possible values based on patterns in other data. Suggested values should be treated as estimates, recorded with their source and confidence level, and reviewed before use.
Where Do Data Quality Problems Come From?
People make mistakes when typing information by hand. A typo, a skipped field, or a wrong format can happen easily, especially when someone is moving fast.
Multiple systems
When a business uses more than one tool to store data, like a CRM and a support tool, each system can end up with its own version of the same information. Over time, these versions stop matching.
Imports
When data gets uploaded from a file, like a spreadsheet, it can carry errors that already existed in that file. If nothing checks the file first, those errors move straight into your system.
APIs
APIs let systems send data to each other automatically. If the sending system has bad data, or if the format changes without notice, the receiving system can end up with broken or missing values.
Migrations
When a business moves data from an old system to a new one, some data can get lost, duplicated, or changed by mistake during the move.
Third-party vendors
Sometimes a business depends on data from an outside company, like a supplier or a partner. If that company's data has problems, those problems become your problems too.
Missing validation
Validation means checking data before it's accepted into a system. If a system doesn't check whether an email looks like an email, or whether a number makes sense, bad data gets in without anyone noticing.
ETL/ELT pipelines
These are the processes that move and reshape data from one place to another. If a step in this process has a bug, or wasn't built to catch errors, bad data can pass through without being stopped.
Schema or business rule changes
An organization decides to change the structure of a table, adding a new field, renaming one, or changing how a value should be recorded. This change only gets made in one system, while other systems keep using the old structure. As data moves between them, it no longer lines up the way it should.
Why Data Quality Matters
Once bad data enters a system, it can affect reports, dashboards, analytics and AI systems.
Business impact of bad data
- Reports show incorrect numbers, which leads teams to make decisions based on a false picture of what is happening.
- Teams lose trust in dashboards because they spend more time checking the data than using it.
- Business processes become slower because people need to manually verify information before taking action.
Technical impact of bad data
- Analytics produce unreliable results because they are based on incomplete or incorrect information.
- AI models learn from incorrect data, so they confidently produce wrong answers, without ever signaling something is off.
- Data pipelines can fail or produce unexpected results when they receive values they were not designed to handle.
- Duplicate or missing records distort everything, from customer counts to revenue totals.
- Troubleshooting becomes more difficult because teams need to trace where the problem started across multiple data steps.
Implementation Example: Our Custom Data Quality Library for dbt
Data quality is a core part of the data platforms we build at MPP Insights. Across projects, we have seen recurring issues in datasets moving through complex pipelines. Addressing them requires more than adding a few validation rules: checks need to run at the right points and account for the database, pipeline design, and business context.
One Example: Our dbt Library
That experience led us to build a data quality library for dbt. This is just one example of how we approach the problem. We work with other tools too, depending on what a project needs.
The database matters here, since different databases run checks in different ways. We work with many types of databases, because every business stores its data differently. ClickHouse is one we use heavily, a database built to handle very large amounts of data at high speed. The library also works with more common databases like PostgreSQL.
How a Check Works
- Each check is defined as a query that returns records that violate an expected rule.
- If the query returns no failing records, the check passes and dbt records a successful result.
- If failing records are returned, dbt flags the check and identifies the affected rows, helping the team narrow down the issue.
- Optionally, failed records can be stored in a separate audit table for later review or analysis in BI tools.
What the Checks Cover
Our library organizes checks into clear groups, so it's easier to understand what's being tested and spot gaps in coverage. These checks catch known, testable issues. Ongoing monitoring and periodic review help catch new or unexpected problems as data and business rules change.
- Missing values: checking whether important fields are empty when they shouldn't be
- Consistency: checking whether the same kind of data is written the same way across records
- Pattern matching: checking whether data follows the format it's supposed to, like an email or a phone number
- Numerical data: checking whether numbers fall within a reasonable, expected range
- Data integrity: checking whether records that depend on each other are actually connected correctly
- Distribution: checking whether data falls within the range you'd normally expect, so unusual spikes or gaps stand out
- Existence of values in sets: checking whether a value belongs to an approved list, like a country code or a status label
- Records count: checking whether the number of records matches what's expected, so nothing is missing or duplicated in bulk
Data Quality Is Part of a Bigger Picture
Good data quality means your data is accurate, complete, and ready to use. But even clean data isn't enough on its own. If nobody knows who is responsible for the data or how it should be managed, the same problems can appear again over time.
This is where data governance becomes important. Without governance, even “good data” can turn to “bad data” again.
What is data governance?
Data governance is the set of rules and responsibilities that decide how data is managed across a business.
- Who owns this data, and who's responsible for keeping it accurate?
- Who's allowed to access it, and who isn't?
- What are the agreed definitions? For example, does "active customer" mean the same thing to sales, marketing, and finance?
- What happens when something changes, who needs to approve it?
These are broader governance and data-management concerns that affect data quality.
How We Help With Data Governance & Quality
At MPP Insights, we help businesses build stronger data foundations through data governance and quality solutions. We create quality checks that identify issues as data moves through your systems, while helping teams define ownership, standardize definitions, establish access rules, and build processes that support better data management over time.
If you're dealing with recurring data quality problems, governance gaps may be part of the reason those issues continue to appear. Reach out to us at welcome@mpp-insights.com to discuss how we can help you build more reliable, well-managed data systems.
