[Learning Center](/learning-center)[Context Engineering](/learning-center/topic/context-engineering) # 6 context engineering mistakes that break enterprise AI ## TL;DR * Swapping in a bigger model rarely fixes accuracy problems. The semantic layer underneath the model is almost always the real cause. * Prompt engineering shapes how a model responds. Context engineering decides whether the model has anything true to respond with. * Most evals start too late. Table selection, not SQL generation, is where answers go wrong. * A data catalog describes what data exists. A semantic context graph encodes the meaning of data. * Effective AI for data needs three layers working together: context, ontology, and memory. * Meaning without a named owner fragments. Finance should define revenue once, and every agent should read from that definition. ## Article Contents * [Mistake 1: Reaching for a bigger model](#mistake-1-reaching-for-a-bigger-model) * [Mistake 2: Engineering the prompt instead of the environment](#mistake-2-engineering-the-prompt-instead-of-the-environment) * [Mistake 3: Evaluating SQL instead of table selection](#mistake-3-evaluating-sql-instead-of-table-selection) * [Mistake 4: Treating the data catalog as the context layer](#mistake-4-treating-the-data-catalog-as-the-context-layer) * [Mistake 5: Buying "context" without ontology and memory](#mistake-5-buying-context-without-ontology-and-memory) * [Mistake 6: Letting meaning fragment with no owner](#mistake-6-letting-meaning-fragment-with-no-owner) * [Three questions to ask any vendor](#three-questions-to-ask-any-vendor) If you want enterprise AI to work on real-world data, you cannot just add more context and hope for the best. The following six mistakes account for most of the failures seen across companies trying to deploy AI on their data. Shawn from [Collate](https://www.getcollate.io/), the team behind [OpenMetadata](https://open-metadata.org/), walks through them in the source video, including work that has helped companies such as Airbus and Yelp improve agent accuracy. ## Mistake 1: Reaching for a bigger model When a demo breaks in production, the first instinct is to swap in a better model. That instinct is almost always wrong. The Spider benchmark tests text-to-SQL on clean academic schemas with about 10 tidy tables. Frontier models score above 90% there. Spider 2.0 uses real enterprise schemas. On that version, GPT-4o drops to 10.1%, and Claude Sonnet gets 10.8%. Same intelligence, same kind of model, and an 80-point collapse. We ran our own test to isolate why. Same model, same question, same data. The only variable was the semantic layer underneath it, specifically the context that tells the model what your tables and columns actually mean. Accuracy went from under 11% to over 76%. The model's capability stayed constant. The same model is either a 10% system or a 76% system depending entirely on what it is told before it writes a query. Time spent evaluating model providers is mostly time spent optimizing the one input that was already good enough. ## Mistake 2: Engineering the prompt instead of the environment If the model swap does not fix it, the next step is usually to adjust the prompt. This one is harder to abandon because it works right up until it matters. Prompt engineering is writing instructions within the context window, including tone, examples, and format. Context engineering is deciding what fills that window in the first place: which facts, which definitions, which tables. One shapes how the model responds. The other decides whether it has anything true to respond with. A better prompt can paper over a missing definition just well enough to pass a demo. You tell the model that net revenue means gross minus four line items, it nails your test question, and everyone moves on. But that definition now lives in your prompt, not in your data. The next agent, the next dashboard, and the next analyst each start from zero. Meaning has to be written down somewhere before a prompt can use it. When your numbers are wrong, missing context is almost always the reason. The [Collate blog](https://www.getcollate.io/blog) and the [OpenMetadata blog](https://blog.open-metadata.org) both return to this distinction: the prompt cannot supply facts the environment never stored. ## Mistake 3: Evaluating SQL instead of table selection Most teams build an eval before shipping, but what are they are actually measuring? A typical warehouse has several tables that could plausibly hold transaction data, forecast data, and revenue variations of all of them. The model has to guess which one answers the question before it ever writes a line of SQL. OpenAI walked through this exact failure recently. Some tables have encrypted customer IDs. Some don't. Some columns are fraud-adjusted. Some aren't. Some tables are pre-filtered. Some aren't. Miss one of those nuances and the answer can be wrong by an order of magnitude. An eval that compares generated SQL against expected SQL starts after that decision is already made. It cannot see the guess that happened upstream. If your eval starts after table selection, you are grading the easy half. Test how the tool decides which table to query. That is the part that actually breaks. ## Mistake 4: Treating the data catalog as the context layer A data catalog inventories what data exists: owner, schema, PII flag. That is necessary and genuinely useful. It is not the same as a semantic context graph, which encodes what the data means and how it connects. The difference matters in practice. Which tables are actually used to calculate loyal customers, and which ones are just similarly named? A catalog cannot answer that. OpenMetadata is built as an open context layer that unifies technical context, business semantics, and organizational memory into a single graph, the structure an AI agent needs to reason over data correctly. The [OpenMetadata documentation](https://docs.open-metadata.org/) covers how that graph is meant to work. ## Mistake 5: Buying "context" without ontology and memory Several vendors now market AI for data platforms. Understanding what they actually ship is important before signing anything. What most of them mean by "context" is schema, data lineage, data quality signals, freshness, and ownership. Those fields describe what your data is. Meaning requires another layer. AI for data actually needs three things that depend on each other. Context is what exists. Ontology is the formal, governed meaning, so an agent asking about customer lifetime value lands on the correct metric rather than the closest-sounding column. Memory is the record of every correction your team has already made, so the next agent does not repeat the last one's mistake. A platform that ships context alone leaves ontology and memory unresolved. It breaks the moment multiple agents start reasoning over conflicting definitions. The [Collate documentation](https://docs.getcollate.io/) is one place that treats those three layers as dependent rather than interchangeable labels. ## Mistake 6: Letting meaning fragment with no owner Walk into most companies, and you will find several definitions of revenue spread across BI tools and warehouses, each defended by a different team. An AI agent querying across all of them may pick one, blend them, and hand you a confidently wrong answer that's nearly impossible to trace back. The fix is federated ownership. Revenue gets defined once, owned by finance, readable by sales, with no quiet overrides allowed. Data ownership has to include ownership of meaning. Meaning without a named owner falls apart the first time two teams give an agent two defensible but conflicting answers. ## Three questions to ask any vendor Before you sign a contract, ask: 1. Does the tool read from the same layer that already powers your data lineage and governance, or does it maintain a separate context store on the side? 2. How does it decide which tables to query before it writes SQL? 3. Does it ship ontology and memory alongside context, or just context with a new label? Then ask your own team two questions: who owns each of your core definitions by name, and where does your eval actually start, before or after table selection? You do not need to fix all six mistakes at once. Focus spending on the layer causing the problem. A glossary was never built to answer that question. The [Collate data platform](https://www.getcollate.io/) is one implementation of a shared semantic layer; the test is whether your stack can answer the three vendor questions above. [ ## Fashion Retailer Mango’s Data Journey with Collate Read the case study ![](data:image/svg+xml,%3csvg%20xmlns=%27http://www.w3.org/2000/svg%27%20version=%271.1%27%20width=%27708%27%20height=%27470%27/%3e)![Mango](/_next/image?url=%2Fimages%2Flearning-center%2Fmango-lc.webp&w=1920&q=75) ](/resources/ebook/mango-case-study) Sign up to receive updates for Collate services, events, and products. ## Share this article [![](data:image/svg+xml,%3csvg%20xmlns=%27http://www.w3.org/2000/svg%27%20version=%271.1%27%20width=%2724%27%20height=%2724%27/%3e)![Share on Twitter](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7) ![Share on Twitter](/_next/image?url=%2Fimages%2Ffooter%2Ftwitter-x-v1.svg&w=48&q=75) ](https://twitter.com/intent/post?url=https%3A%2F%2Fgetcollate.io%2Flearning-center%2F6-context-engineering-mistakes-that-break-enterprise-ai%3Fref%3Dtwitter-share)[![](data:image/svg+xml,%3csvg%20xmlns=%27http://www.w3.org/2000/svg%27%20version=%271.1%27%20width=%2724%27%20height=%2724%27/%3e)![Share on LinkedIn](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7) ![Share on LinkedIn](/_next/image?url=%2Fimages%2Ffooter%2Flinkedin-v1.svg&w=48&q=75) ](https://www.linkedin.com/feed/?linkOrigin=LI_BADGE&shareActive=true&shareUrl=https%3A%2F%2Fgetcollate.io%2Flearning-center%2F6-context-engineering-mistakes-that-break-enterprise-ai) Ready for trusted intelligence? See how Collate helps teams work smarter with trusted data [Get Started](/welcome)[Contact Us](/contact-sales)