Tags

# Data Engineering

[

![How to Get Started with Data Quality as Code in Your Pipelines](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)

![How to Get Started with Data Quality as Code in Your Pipelines](https://cdn.hashnode.com/res/hashnode/image/upload/v1768580572815/d515037b-b61f-446a-9d08-871f5ac000ae.png)

## How to Get Started with Data Quality as Code in Your Pipelines

In Part 1, we explored why traditional data quality testing happens too late—after bad data has already reached production. Data Quality as Code shifts validation left, letting you catch issues during transformation before they propagate downstream. ......

](/blog/how-to-get-started-with-data-quality-as-code)

[

![Introducing Data Quality as Code: Validate Data Before It Lands, Not After](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)

![Introducing Data Quality as Code: Validate Data Before It Lands, Not After](https://cdn.hashnode.com/res/hashnode/image/upload/v1768579612660/a7b5d451-7df1-42b7-9e01-cd5393141645.png)

## Introducing Data Quality as Code: Validate Data Before It Lands, Not After

Every data team knows the scenario: By the time quality tests catch errors, bad data has already reached production and hit user dashboards and business reports. The problem isn't that organizations don't test their data; it's that they test it too l......

](/blog/introducing-data-quality-as-code)

[

![Real-time Metadata Ingestion: APIs vs Kafka Explained](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)

![Real-time Metadata Ingestion: APIs vs Kafka Explained](https://cdn.hashnode.com/res/hashnode/image/upload/v1768240858337/a1a7caf9-789f-43be-851c-3edd5885db06.png)

## Real-time Metadata Ingestion: APIs vs Kafka Explained

Metadata is the foundation of modern analytics and AI. When schemas change, new tables appear, or ownership shifts, those changes ripple immediately through dashboards, models, and pipelines. If metadata lags behind reality, teams lose trust and syst......

](/blog/real-time-metadata-ingestion-apis-and-kafka-explained)

[

![2026 Predictions: Why Semantics Will Determine AI Success](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)

![2026 Predictions: Why Semantics Will Determine AI Success](https://cdn.hashnode.com/res/hashnode/image/upload/v1767630717816/2bda9e63-2d83-446c-85e3-ab37f124acc8.png)

## 2026 Predictions: Why Semantics Will Determine AI Success

As we reach the end of 2025, a pattern has emerged which is hard to ignore. AI adoption is everywhere, but companies are struggling to see large-scale and repeatable value creation. In its 2025 State of AI survey, McKinsey reported that while enterpr......

](/blog/2026-predictions-why-semantics-will-determine-ai-success)

[

![Introducing Custom Governance Workflow Builder](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)

![Introducing Custom Governance Workflow Builder](https://cdn.hashnode.com/res/hashnode/image/upload/v1766467890967/804628f2-f47d-487c-bab7-9ed390e502cd.png)

## Introducing Custom Governance Workflow Builder

Data teams struggle with governance that's either too rigid or too manual. Pre-built workflows don't fit your business needs, while manual processes don't scale with your teams. This makes enforcing governance consistently across hundreds of assets a......

](/blog/introducing-custom-governance-workflow-builder)

[

![Announcing Collate 1.11](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)

![Announcing Collate 1.11](https://cdn.hashnode.com/res/hashnode/image/upload/v1765469415636/da73985c-262d-48d2-a99e-da8f43c1c650.png)

## Announcing Collate 1.11

We're excited to announce Collate 1.11, the latest release of our managed OpenMetadata service—and a significant leap forward in AI-powered semantic intelligence. This release introduces: AskCollate - our conversational AI assistant that brings natu......

](/blog/announcing-collate-111)

![](data:image/svg+xml,%3csvg%20xmlns=%27http://www.w3.org/2000/svg%27%20version=%271.1%27%20width=%2716%27%20height=%2716%27/%3e)![arrow-left](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)

![arrow-left](/images/learning-center/left-arrow.svg)

12

![](data:image/svg+xml,%3csvg%20xmlns=%27http://www.w3.org/2000/svg%27%20version=%271.1%27%20width=%2716%27%20height=%2716%27/%3e)![arrow-right](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)

![arrow-right](/images/learning-center/right-arrow.svg)