Module 5 of 6 · 10 min

Data Quality Basics

How quality is tested and expressed, so you can judge whether an asset is fit for your purpose.

Why this matters

A table that works for a trend chart may be unusable for financial reporting. Quality tests and their results give you the evidence to decide whether an asset fits your use.

Before you start

Tests are the evidence of quality

An asset is trustworthy to the extent that someone has written tests expressing what correct means for it, and those tests pass. An asset with no tests is unmeasured, and you should treat it that way.

Freshness is usually the test that matters most

A pipeline stops running and leaves a table that looks normal and is out of date. A freshness test catches that case, which no schema or null check will.

Tests run on a schedule, so recency of the result matters too

A passing test tells you the asset was correct when the test last ran. If the test runs weekly and the data lands hourly, a green result is weaker evidence than it looks. Read the result and the schedule together.

The interactive demo

Work through the demo below. It runs in your browser with nothing to install. Watch for:

  • Creating a data quality test
  • Creating a pipeline to run it
  • Scheduling the test and pipeline to check for data freshness

This demo is also available on its own at the Learning Center.

After this module you should be able to

  • Read a quality result and say what it does and does not prove
  • Explain why a freshness test catches the most common failure mode
  • Decide whether a given asset is fit for a specific purpose, and justify it