A Semantic Layer for Agents: Keep Business Metrics Consistent

Why an agent needs a business contract

A business question often sounds simpler than the data model behind it. “What was net revenue last quarter?” may depend on which transactions count, how refunds are treated, which date defines a quarter, how currencies are converted, and whether the question concerns booked or recognized revenue. People familiar with the business may know these conventions. A language model cannot safely infer them from column names.

A semantic layer is the governed translation between business concepts and underlying data. It gives a system a defined way to interpret terms such as “net revenue,” “active customer,” or “quarter,” and connects those definitions to approved data structures and calculation rules. An AI agent should use that translation rather than inventing its own interpretation from raw warehouse tables.

The key idea is not that a semantic layer makes every answer correct. It narrows what the agent is allowed to mean, ask and return. A reliable design also needs a tool contract that limits the agent’s query options, an access contract that constrains whose data can be seen, and tests that compare answers with known business expectations.

These contracts are related but distinct. The metric definition answers, “What does this measure mean?” The tool contract answers, “What operations may the agent request?” The permission contract answers, “Which records may this user or agent see?” Confusing one for another creates gaps: a correct metric may still expose the wrong customer records, while a perfectly restricted query may calculate the wrong measure.

What the semantic layer contributes

Consider a physical warehouse with tables for invoices, invoice lines, subscriptions, customers and currency rates. A business user does not want to choose a join path or remember which invoice status represents a finalized sale. They want to ask a question using familiar terms and receive an answer with a traceable definition.

A semantic layer represents those terms in a form that can be translated into data operations. A metric is a named calculation, such as net revenue. A dimension is a way to group or filter that calculation, such as product line, customer segment or fiscal month. A join path is an approved relationship between data entities. A grain describes what one row represents, such as one invoice line or one customer per month.

These definitions help prevent common errors. Summing a distinct customer count across months can overstate the number of unique customers. Joining invoice lines to multiple matching customer records can multiply revenue. Using an invoice creation date in place of a revenue-recognition date can move activity into the wrong reporting period. Each error can produce a tidy number with a plausible explanation.

A semantic layer should therefore express more than labels. For each metric, document the calculation, included and excluded records, time basis, unit, permitted dimensions and known caveats. For each dimension, specify its source and expected grain. These details give both the agent and a human reviewer something concrete to validate.

The layer also provides a place to resolve vocabulary. A sales leader might say “new business,” while the warehouse uses “first contract” and finance reports “new recurring revenue.” Those phrases may overlap without meaning the same thing. Treating them as synonyms without an approved mapping is a business decision disguised as natural-language processing.

A useful definition names the metric precisely and makes ambiguity visible. If “active customer” means a customer with a paid transaction during the selected period, say so. If a separate operational definition includes trial users, give it a different name. When two teams need different measures, preserve both definitions and identify their owners instead of silently picking one.

Three contracts between question and answer

The first contract is the metric contract. It records the business meaning: formula, grain, time basis, filters, allowed groupings, units and ownership. The agent should select from these definitions, not write a new formula because a phrase resembles a column name.

The second is the tool contract. This is the boundary around what a model can ask a data service to do. A narrow contract might let the agent select an approved metric, choose from approved dimensions, supply bounded filters and request a result limit. It should not let a free-form answer turn into arbitrary warehouse SQL merely because the user phrased a request confidently.

The third is the permission contract. It binds the question to an authenticated user or service identity and defines which records that identity may access. The identity and its attributes must come from a trusted part of the application flow, not from a user prompt or a model-generated filter. Otherwise a request such as “show me every region” can become a way to override intended visibility.

These contracts should be checked in sequence. First determine whether the requested concept has an approved meaning. Then validate the requested operations and parameters. Next apply access rules for the calling identity. Only after those checks should a query be generated and run. The returned result needs its own validation before the agent turns it into prose.

flowchart TD
  accDescr: Workflow stages and decisions: User question and identity, Resolve approved metric, Validate tool request, Apply access policy, Run bounded query, Check result and answer, Clarify or decline. The adjacent text explains the conditions and exceptions.
  accTitle: A Semantic Layer for Agents — Keep Business Metrics Consistent workflow
    U["User question and identity"] --> M["Resolve approved metric"]
    M --> T["Validate tool request"]
    T --> P["Apply access policy"]
    P --> Q["Run bounded query"]
    Q --> V["Check result and answer"]
    M -->|"No approved meaning"| H["Clarify or decline"]

The diagram distinguishes interpretation from execution. If no approved meaning exists, the system should ask for clarification or decline to calculate, rather than fabricating a definition. If a tool request is invalid or the identity lacks access, the request should stop before query execution. A returned number also needs checking: a syntactically valid query can still produce an empty, incomplete or unexpected result.

A useful implementation pattern is to make each stage return a structured outcome: accepted, rejected, needs clarification, or failed. That avoids forcing every condition into a natural-language answer that sounds equally confident. It also makes operational review more specific: teams can distinguish missing business definitions from permission denials and warehouse failures.

The labels in this flow are design concepts, not claims about a particular vendor API. An organization can implement them with its existing data and application components, provided each boundary is explicit and testable. Avoid granting the model a broad capability simply because the first prototype is easier to assemble that way.

Designing a narrow tool contract

A tool contract should expose business choices rather than warehouse internals. For example, a request could identify a defined measure, a reporting period, a permitted grouping and a small set of filters. The application can then validate those choices and translate them through its semantic layer. The exact request format depends on the implementation; the important point is that the allowed fields and values are constrained.

An illustrative request might express “net revenue by product line for the previous fiscal quarter,” with an explicit measure identifier, time window and grouping. It should not rely on the model to supply a table name, invent a join, choose a hidden currency conversion rule or decide which account statuses count. Those decisions belong in approved definitions and application logic.

Validation should cover both shape and meaning. Shape checks include whether required fields are present, whether filters use allowed operators, and whether values match expected types. Meaning checks include whether the selected dimension is valid for that metric, whether the period is supported, and whether the requested detail would expose records that should only be available in an aggregate.

Bound the request before execution. Useful limits can include a maximum time range, a maximum number of groupings, an allowed result size and a timeout appropriate to the product experience. These controls are not substitutes for efficient data modeling, but they reduce the impact of accidental broad queries and make the tool’s behavior easier to predict.

Do not let a request parameter quietly change the definition. If a user asks for “net revenue excluding refunds,” and the approved net-revenue metric already excludes refunds, the system should explain the overlap rather than removing refunds twice. If the organization supports a distinct alternative measure, the agent should select that named definition and disclose the distinction.

The tool should return enough context for the agent to explain the result without reconstructing it. That may include the metric name, period, grouping, applied filters, result status and a reference to the approved definition. Keep this context concise and avoid returning sensitive source fields merely to help generate a fluent explanation.

A missing definition is a product behavior, not an invitation to guess. If someone asks for “quality revenue” and no such measure is approved, ask what they mean or route the request for definition. If the phrase maps to multiple metrics, show the ambiguity in plain language and request a choice. A short clarification is often more useful than a precise-looking answer based on an unstated assumption.

Permissions are not a metric feature

Metric definitions and access controls solve different problems. A semantic layer can tell the system how to calculate a measure without establishing that every person may see every row contributing to it. Treating the presence of a business metric as permission to query all of its underlying data is an unsafe shortcut.

For Apache Superset, row-level security filters apply clauses to generated queries for configured datasets and subjects. Teams should separately validate SQL Lab and API access; embedding alone is not authorization. See the Apache Superset security documentation and build permission checks around the actual access paths used by the product.

Start with the identity chain. Record which user initiated the request, what authenticated application context is passed to the data service, and where the resulting identity-to-data mapping is enforced. A model must never be the authority that decides which user it represents. Do not accept a region, account list or role supplied in natural language as proof of access.

Then identify every route to the data. An application may query through a semantic service, a BI interface, a SQL console or an API. Each path needs an explicit decision about whether it is available to the agent and how access is enforced. Testing only the ordinary dashboard route does not establish that another route has equivalent behavior.

For embedded analytics, distinguish presentation from authorization. Showing a user a chart inside an application does not, by itself, prove that the underlying query is restricted to that user’s permitted records. If an embedded experience is part of the product plan, define where the trusted identity enters, how its scope is enforced and what happens if that context is missing or malformed. Teams planning embedded analytics should make this boundary part of the architecture rather than treating it as a late interface detail.

Permission tests need both positive and negative cases. A positive case confirms that an authorized user can see expected rows. A negative case confirms that a user outside the intended scope cannot retrieve those rows through every enabled path. Include requests that try alternate wording, broad filters and direct access routes; access should not depend on whether the model happens to phrase a query carefully.

For a deeper treatment of generated-query permission and wrong-answer tests, use Agent-Generated SQL: A Test Pack for Permissions and Wrong Answers. This article focuses on how semantic definitions and tool boundaries fit together, rather than duplicating that test-pack scope.

Worked example: revenue by product line

The following is a hypothetical design for a subscription business. It is not a report of an implemented system. Assume finance defines net revenue as recognized subscription charges for the selected period, less refunds and credits, converted to the reporting currency using an approved daily rate. Trial accounts and internal test accounts are excluded. Revenue is grouped by the product line attached to the recognized charge.

The request is: “How much net revenue did each product line contribute last quarter?” Before querying, the application must resolve “last quarter” against the organization’s reporting calendar. It must also determine whether “contribute” means absolute net revenue or each product line’s percentage of the total. If the product cannot infer that choice from an agreed default, it should ask for clarification.

Suppose the user confirms absolute net revenue and the system resolves the requested period to the prior fiscal quarter. The metric contract selects the approved definition, its recognition-date basis, the refund and credit treatment, the currency conversion convention, and the exclusion rules. The tool contract allows grouping by product line and blocks unsupported dimensions such as individual customer email.

The permission contract then evaluates the caller. For this illustrative scenario, assume the caller is a regional manager restricted to a named set of markets. The query must apply that restriction through a trusted identity context, not by asking the agent to add a region filter. If the metric is unavailable to that identity, the system must deny it rather than return a result with an empty-looking filter.

A successful result should include the period, currency, grouping and applied business definition alongside the values. The agent can then say what it calculated and identify any meaningful limitation, such as a product line with no qualifying activity. It should not claim a trend, cause or forecast when the request only produced a period total.

Now consider a counterexample. A user asks for “revenue from customers we won last quarter.” The phrase might mean cash collected from newly acquired customers, recognized revenue from new contracts, or recurring revenue added through new sales. These definitions can lead to different answers. A semantic layer helps only if the distinctions are represented and the agent knows when it must ask the user to choose.

A second counterexample concerns joins. If one customer can have multiple active product-line records, joining each invoice line to all of them may duplicate amounts. The metric’s allowed dimensions and join path should prevent that result, while a reconciliation check should compare the aggregate total with a trusted finance report for the same period and definition.

An illustrative accuracy check could use a small, reviewed set of questions with expected results. For example, five approved questions might cover a total, a time comparison, a permitted grouping, an ambiguous term and a user outside the allowed data scope. The purpose is not to claim that five examples establish accuracy; it is to make failure modes explicit before expanding the test set.

For each expected answer, record the question, caller context, selected metric, period, filters, expected result or expected refusal, and the reviewer’s rationale. Compare numerical answers using a tolerance that finance has approved for rounding and conversion effects. A permission test should fail if restricted rows appear, even when the aggregate number happens to look plausible.

Testing meaning, access and answer quality

A useful test pack separates at least three questions. Did the system choose the intended business definition? Did it enforce the caller’s access scope? Did it describe the returned result accurately? Passing one dimension does not imply passing the others.

For metric accuracy, test questions with different surface wording that should map to the same definition. Include terms that look similar but should map to different metrics, and requests whose definitions are absent. Review the selected metric and parameters as well as the final number; an accidentally correct total can conceal an invalid calculation path.

For access, create representative identities and data scopes. Test an allowed query, a denied query, an attempted scope expansion and each enabled query route. Record which rows or aggregates should be visible, and inspect the result rather than treating a friendly denial message as proof that no data was retrieved.

For answer quality, compare the generated explanation with the result metadata. Check that it names the period and grouping correctly, does not omit a material filter, does not turn correlation into causation, and does not add a value absent from the returned result. When the query failed or the result is incomplete, the answer should preserve that status rather than present a clean summary.

A practical review record can use a compact table:

Field	What to capture
Question and identity	User wording and trusted caller scope
Intended meaning	Approved metric, period and grouping
Expected outcome	Value, clarification, denial or failure
Actual outcome	Selected definition, query status and returned result
Review decision	Correct, incorrect or needs business clarification
Follow-up	Definition change, tool-boundary fix or access investigation

Keep “needs business clarification” separate from “model error.” If finance has not settled whether a measure includes one-time fees, no model can resolve that organizational decision reliably. Likewise, a permission denial may be correct even when a user finds it inconvenient. The review record should identify which owner can resolve each kind of issue.

Changes to metric definitions deserve regression checks. A revised refund rule can affect totals across many periods; a new dimension relationship can change grouping results. Before enabling the change for agent queries, compare representative outputs under the old and proposed definitions, document the intended difference, and verify that unrelated metrics did not change unexpectedly.

Operational failure paths

A semantic layer does not eliminate data quality or availability failures. It can faithfully apply a definition to stale, incomplete or misclassified data. If freshness matters to a decision, return the data’s relevant freshness status and define when the answer must be withheld or labelled provisional.

Failures should be legible at the stage where they occur. An unrecognized business term calls for clarification. An invalid grouping calls for a constrained rejection. A denied identity calls for an access response. A warehouse timeout calls for a service failure, not a fabricated number. A result that fails reconciliation should be held for review or clearly marked according to an agreed policy.

A common operational mistake is to retry a failed request with a broader query. That can turn an ordinary timeout into an expensive scan or bypass the intended limits. Retry behavior should preserve the same metric, identity scope and bounds. If the service cannot complete within the defined limit, report the failure and let the user choose a narrower request or try later.

Another failure arises when the definition changes but the explanation does not. If “net revenue” now excludes a category of credits, old answers and new answers may differ even for the same period. Include a definition version or effective date in internal review records so analysts can explain why the result changed. The user-facing product can communicate meaningful changes without burdening every answer with implementation details.

The deployment environment matters as well. If Superset is part of the design, configure the required identity and row-level rules for the datasets and subjects in use, then test each route the agent can reach. The production row-level security patterns for Apache Superset and production Helm values for Apache Superset offer related implementation context; neither replaces application-specific verification of the query paths and identity flow.

Treat monitoring as a way to find review candidates, not as proof of correctness. Useful signals include repeated clarification requests, rejected dimensions, empty results, failed reconciliations, permission denials and changes in which metrics users request. Each signal needs interpretation: a rise in denials could indicate an attempted bypass, but it could also follow a legitimate change in job responsibilities.

A staged path to a useful first release

Begin with a narrow question set. Select a small number of measures that have clear owners, stable definitions and a meaningful business use. Prefer questions whose expected results can be checked against an existing trusted report or calculation. Leave ambiguous, highly sensitive or poorly reconciled measures out of the first scope.

Write the metric contracts before connecting an agent. For each candidate, specify its grain, formula, date basis, filters, currency treatment, allowed dimensions and known caveats. Ask the business owner to resolve competing interpretations. If the owners cannot agree on a definition, keep that measure unavailable rather than presenting a provisional interpretation as settled fact.

Next, specify the tool contract and permission model. List the operations the agent may request, the fields it may set, the dimensions it may use and the limits enforced before query execution. Document which authenticated identity supplies access scope and which paths are explicitly out of bounds. Keep a human-readable rejection reason for each boundary so product teams can improve confusing behavior without weakening controls.

Build a test set from real business questions, edited to remove unnecessary personal or confidential detail. Include questions that should answer, questions that should clarify, questions that should be denied and questions that should fail safely. For each, agree on the expected definition and result before looking at what the model returns.

Release to a constrained audience and review representative interactions. Record enough structured information to diagnose a decision—question intent, selected measure, allowed parameters, access outcome and result status—while applying the organization’s ordinary data-handling rules to logs. Avoid retaining more sensitive query content than the review process needs.

Expand only when the observed errors lead to concrete fixes. A wrong join points toward a relationship or grain problem. A correct calculation with a misleading explanation points toward answer construction. An unauthorized row points toward an access boundary and should block expansion until resolved. An unsettled term points toward business definition work, not a prompt tweak that hides the disagreement.

Teams can also keep product analytics separate from service inquiries when evaluating how people arrive at the experience. That is a distinct measurement concern from metric correctness; see Measuring AI-Search Referrals Without Mixing Product and Services Funnels for that separate topic.

A printable decision worksheet

Use this worksheet in a design review or release decision. It is intended to be copied into the team’s working materials; it is not a substitute for implementation-specific tests.

Metric meaning

• The business owner has approved the metric name, formula and grain.
• The time basis, fiscal calendar, currency treatment and exclusions are explicit.
• Allowed dimensions and known limitations are documented.
• Similar terms map to distinct definitions where their meanings differ.
• The system has a clarification path for terms with no approved definition.

Tool boundary

• The agent can request only supported measures, dimensions, filters and operations.
• Request parameters are checked for valid form and valid combinations.
• Time range, result size and execution limits are defined for this use case.
• Missing or invalid parameters produce a clear rejection or clarification.
• Results include enough context to explain the calculation without exposing unnecessary fields.

Access and verification

• The caller identity comes from a trusted application context.
• Access is checked for every enabled route to the data.
• Tests cover both authorized results and attempts to exceed the caller’s scope.
• Expected answers, clarifications, denials and failures are recorded with reviewer rationale.
• A business reviewer has approved the release criteria and any numerical tolerance.

Operations

• Stale or incomplete data has a defined treatment.
• Query failures cannot silently widen scope or remove access constraints.
• Definition changes receive regression checks and an effective-date record.
• Review signals have owners who can distinguish business ambiguity from technical failure.
• The team knows what condition pauses expansion or requires rollback.

Use the worksheet to decide whether the system is ready for a specific question set, not whether an abstract “AI analytics” capability is ready in general. A narrow release with clear definitions and visible failure behavior is more informative than a broad launch whose errors cannot be attributed to metric meaning, query construction or access control.

The practical standard

A semantic layer makes agent answers more consistent when it is the agent’s route to business meaning, not simply another catalog alongside raw tables. The business definition needs to be explicit, the tool needs constrained choices, and the caller’s access needs to be enforced independently of the model’s interpretation.

The standard for a useful answer is not fluent wording. It is a result that can be traced to an approved metric, evaluated against an expected outcome, and shown only to an authorized caller. When one of those conditions is missing, the system should clarify, deny or report a failure instead of filling the gap with confident prose.

For organizations translating this architecture into a product or platform plan, PADISO’s embedded analytics service is a relevant next step for discussing implementation needs. The starting point remains a concrete set of business questions, definitions and access paths—not a general promise that an agent can answer every question.