Microsoft Foundry Agent Architecture: Separate Runtime, Tools and Data

Table of contents

• 1. Start with the boundaries, not the product diagram
• 2. Define the reference architecture
• 3. Assign responsibility at each boundary
• 4. Keep data authorization outside the agent’s instructions
• 5. Design tools as controlled business interfaces
• 6. Map identity and network paths without conflating them
• 7. Work through a hypothetical service-operations design
• 8. Plan for failure across component boundaries
• 9. Use the architecture worksheet as a release artifact
• 10. Sequence implementation and next steps

1. Start with the boundaries, not the product diagram

An enterprise agent architecture is a set of responsibilities and permitted call paths, not simply a diagram of cloud services. A useful design distinguishes the application that starts a task, the agent runtime that coordinates it, the model endpoint that generates responses, the tools that perform work, and the data services that return information. These components may be operated by different teams. The architecture must make their boundaries visible before the teams integrate them.

This separation prevents a common category error: treating an agent as if it were both an application identity system and a complete authorization layer. The runtime can coordinate a request, but the system that serves a record or performs a business action must still decide whether that request is allowed. A prompt may describe what the agent should do; it cannot establish a user’s entitlement to a customer record or approve a payment.

The design goal is not to create the largest possible number of services or handoffs. It is to put each decision at the layer with enough context and authority to make it safely. The application knows which person or workload initiated a task. A data service can evaluate access to a particular record. A business system can enforce its own action rules. The platform team can define which runtime and network patterns are supportable.

Begin with the business journey. Write down what a user asks, what information must be retrieved, what actions might follow, and which existing systems own those records and actions. Then draw only the components needed to carry out that journey. If the picture does not show who decides access or where a rejected call stops, it is not yet a useful architecture.

Platform selection is a related but separate decision. The broader enterprise agent platform comparison can help frame that choice. This guide assumes Microsoft Foundry is the selected platform and focuses on the responsibilities that remain after selection: runtime, tools, data, identity context, and the paths connecting them.

2. Define the reference architecture

The diagram below is a logical reference design, not a claim that every deployment uses this exact topology. It shows the major trust boundaries and the direction of work. For each arrow, record the caller, the target, the information passed, the authorization decision, and the team responsible for resolving a failure.

flowchart TD
accTitle: Logical boundaries in a Microsoft Foundry agent architecture
accDescr: An enterprise application sends a task to an agent runtime. The runtime calls a model endpoint and a controlled tool gateway. The gateway connects to an enterprise system, while a separate data service returns tenant-scoped information to the application workflow.
    A["Enterprise application"] --> B["Agent runtime"]
    B --> C["Model endpoint"]
    B --> D["Controlled tool gateway"]
    D --> E["Enterprise system"]
    B --> F["Tenant-scoped data service"]

Read the application-to-runtime arrow as the entry to the agent workflow. The application should establish the caller context that the rest of the workflow needs. Decide whether the interaction is on behalf of a person, a background process, or a combination, and make that distinction explicit in the design. Do not let a tool infer the original user from an arbitrary prompt field when the application can establish that context through its own trusted path.

The runtime-to-model arrow represents the generation step, not a grant of access to business records. The model may help interpret a request or choose among available actions, but the model response does not become an authorization decision. The runtime-to-tool arrow should lead to a deliberately scoped interface. That interface can validate the requested operation and pass a well-formed request to the system that owns the business data or action.

The separate data-service path is a design choice, not a requirement to create a new service for every deployment. It makes the data authorization boundary explicit. In an existing application, a well-controlled service may already perform this role. What matters is that the service receiving a data request can evaluate the caller’s rights and the record’s tenant or business context without trusting an instruction generated during the conversation.

Keep the diagram logical until the teams have confirmed the actual deployment pattern. Foundry supports different agent deployment approaches: hosted agents package custom code as containers, while prompt agents use declarative prompts and tools. Those approaches affect implementation and operations, but neither changes the need to identify the runtime’s permitted calls and the data or action owner’s authorization decisions (Microsoft’s overview of hosted agents). For a focused deployment-model discussion, see the comparison of Foundry prompt agents and hosted agents.

A useful diagram review follows each path in both directions. For a request, ask where its identity and tenant context originate, which component can alter them, and which service checks them. For a response, ask which data can return, whether the application can associate it with the right user and task, and what happens if the response is incomplete or delayed. This turns boxes and arrows into testable design questions.

3. Assign responsibility at each boundary

A boundary without a named owner tends to become a shared assumption. The application team may assume the platform team validates users; the platform team may assume the data owner restricts records; the data owner may assume the agent never asks for an out-of-scope record. Convert those assumptions into explicit decisions. An accountable owner does not have to implement every component, but should be able to state what it permits, what it rejects, and how that behavior will be verified.

The following matrix is a practical starting point. Adapt role names to your organization. A small engineering group may have one person covering several rows; a larger organization may split them among application, AI platform, data, security, network, and system teams. The important distinction is ownership of a decision, not the number of teams in the table.

Review each row for a decision that is easy to say but difficult to prove. For example, the phrase “the runtime can call the customer API” leaves open which operations are possible, what identity is used, and how the API identifies the customer. Replace it with a precise statement such as: “The workflow may request a read-only customer summary using the authenticated application context; the customer service checks the caller’s access before returning records.” That is still a design statement, so it must be validated against the actual implementation.

For every cross-team boundary, agree on an interface contract. Keep it small enough to review: purpose, allowed operations, required fields, rejected conditions, expected response shape, timeout behavior, and owner. Add a version or change-control convention where independent teams will update the interface. Without a shared contract, one team can make a technically valid change that silently widens access or breaks the downstream workflow.

The contract should describe what happens when information is missing. If a request lacks a tenant identifier, does the application reject it, or does a service resolve the tenant from trusted caller context? If a requested operation is outside the allowed set, does the interface reject it before reaching the business system? These are architecture decisions because they determine where an invalid request stops and which team can diagnose it.

Keep the matrix close to the actual deployment record rather than treating it as a workshop artifact. When a new tool is introduced, the matrix should show which owner approves its operation set and which system enforces access. When a data source moves, the data owner should confirm that the new service preserves the intended authorization boundary. A current matrix can shorten reviews because teams can compare a proposed change with the decisions already in force.

4. Keep data authorization outside the agent’s instructions

For a multi-tenant system, a prompt that says “only show this customer’s records” is not a tenant isolation control. It is an instruction to the agent, not an authorization check performed by the service that returns data. The service that owns or serves the records needs enough trusted context to decide whether the caller can access them. This remains true if the agent is accurate in ordinary cases: authorization must also hold when a request is malformed, manipulated, or produced under an unexpected conversation state.

A useful data contract separates caller context from user-supplied content. Caller context should come from a trusted application or service path. A user may ask for a different account, include an identifier in a message, or provide a document that mentions another tenant. Treat those values as requested inputs, not proof of entitlement. The data boundary should either verify that the caller is authorized for the requested record or reject the lookup.

Design tests around changes in one variable at a time. First, keep the caller fixed and substitute a different tenant identifier in the request. Second, keep the requested record fixed and change the caller to one without access. Third, include a misleading identifier in retrieved text or other untrusted content and verify that it cannot redirect the data lookup. For each test, record which service rejects the attempt. A denial that depends only on the model choosing not to comply is not evidence that the data boundary is enforcing access.

Do not make every data failure look like an empty result. An empty result can mean no matching record, an unauthorized caller, an unavailable service, or a malformed request. Those outcomes may need different handling by the application and operations teams. Define which distinctions the service can return safely, and avoid exposing sensitive details to a caller who should not see them. Internally, retain enough diagnostic context to identify the failing boundary without recording unnecessary sensitive content.

A practical release test includes a valid request and a negative request for the same class of data. The valid case proves the design is usable; the negative case proves a nearby forbidden path is blocked. Add a test in which a user changes a visible identifier but not their authenticated identity. If the response changes to include another tenant’s records, the boundary is misplaced or incomplete. Fix the enforcement point rather than trying to refine the prompt as a substitute.

Identity architecture is its own substantial subject. Keep this article’s concern narrow: establish which component passes caller context and which service uses it to decide access. For the deeper treatment of roles and permitted actions, continue with the guide to Entra identity for AI agents.

5. Design tools as controlled business interfaces

A tool is an interface between the agent workflow and an operation or source of information. It should be designed like a business API: a named purpose, a bounded set of operations, inputs that can be validated, outputs that can be interpreted, and a clear owner. The agent should not receive a general-purpose connection merely because a broader interface is easier to implement. Every operation that can read, write, approve, or trigger a downstream process changes the architecture’s risk and operational burden.

Start by writing the user task in business terms and breaking it into operations. A service desk workflow might look up a case, add an internal note, or request a status change. Those are separate operations even if one backend supports all three. Separate them in the tool contract when they require different rights, validation, or operational handling. A workflow that can read case summaries should not implicitly gain permission to close cases or change customer ownership.

Give each operation a narrow input schema. For example, a case lookup might accept a case reference and a requested field set, while the service resolves the caller’s access using trusted context. A status-change request might accept the case reference, target state, and a business reason. The tool should validate allowed values and reject unexpected fields. These are proposed design patterns, not claims about a particular Foundry feature; implement them in the component that owns the interface.

Where a tool can cause a material external change, separate proposing an action from executing it. The approval should refer to the exact action and payload that will be executed, not a general statement such as “the agent may update this case.” If the payload changes after approval, require a new decision. Define an expiry so an old approval cannot authorize a different action later. After execution, verify the result in the system that owns the business state rather than treating the agent’s completion message as proof that the action succeeded.

This design also clarifies the role of a tool gateway. A gateway can provide a controlled integration boundary when several workflows need a consistent interface, but it must not become an unreviewed bypass around the system of record. Record what the gateway validates and what the downstream system must still enforce. If the gateway can call a privileged operation, its own owners need a clear process for reviewing changes to the operation set and for tracing which workflow requested each call.

Microsoft describes Toolbox as exposing curated tools through an MCP-compatible endpoint and notes that tool versions and permissions need governance (Toolbox overview). The architecture implication is to assign an owner for each exposed tool, review the version and permission set, and test the operations that should be denied. Do not infer that a curated listing by itself enforces your enterprise authorization policy.

Before adding a tool, ask whether the operation is necessary for the user journey, which system owns the underlying data or action, what identity and tenant context the target evaluates, and how you will know the operation completed. If the answer to the last question is “the model said it worked,” the workflow needs an independent business-state check or a clear unresolved outcome.

6. Map identity and network paths without conflating them

Identity and networking answer different questions. Identity concerns who or what is making a request and what that caller may do. Networking concerns whether a component can reach another component over an allowed path. A successful connection does not prove that the caller is authorized; an authorized identity does not prove that the required route is available. Treat them as separate design views and then test their intersection.

For each arrow in the reference diagram, capture both views. On the identity side, record the calling component, the context passed onward, and the service that makes the access decision. On the network side, record the source, destination, direction, intended route, and team that confirms the route. This keeps a connectivity change from being mistaken for a permission change and prevents an identity configuration from being treated as a substitute for network review.

Microsoft’s networking guidance distinguishes private inbound access from outbound isolation: private inbound access alone does not isolate egress, and tool compatibility needs checking for the chosen configuration (Foundry networking options). The practical design response is to map both entry and exit paths. List the destinations a workflow needs to reach, identify which team approves those paths, and confirm that the selected tools work with the intended network configuration.

Do not expand this reference architecture into a detailed private-network implementation plan unless that is the current decision. The important ownership questions here are who approves the routes, who validates connectivity, and which team owns a failure when the runtime cannot reach a required dependency. The deeper discussion of network topology and outbound paths is covered in the private networking guide for Foundry agents.

A clean review exercise tests a valid identity over a valid route, then tests a denied identity over that same route. Separately, confirm that the intended identity cannot use an unintended route. This distinguishes authorization failures from connectivity failures. Capture the expected result for each test; otherwise, an error can be misclassified as a successful block or a temporary network issue.

7. Work through a hypothetical service-operations design

Consider a hypothetical North American service company supporting several business customers. Its operations team wants an agent to summarize a support case, find relevant service history, and propose a follow-up update. The company has a case system, an asset database, and an application used by employees. These details are illustrative; they describe a design exercise, not a deployed system or a PADISO customer result.

The application begins by establishing which employee is signed in and which customer account the employee is working on. It sends a task to the agent workflow with a case reference and the minimum context needed to locate the case. The case and asset services do not trust a customer identifier supplied only in conversational text. They independently check whether the caller may access the requested case and associated assets.

The initial version exposes two read operations: retrieve an allowed case summary and retrieve a limited set of related asset details. The workflow can then produce a draft update, but it cannot directly change the case state. This boundary is deliberate. It gives the operations team a useful retrieval and summarization journey without combining read access and state-changing authority in the first release.

Suppose the business later requests an option to add a follow-up note. The team defines a separate write operation with a case reference, exact note text, and a reason category. The case system checks that the caller can update that case. The application presents the proposed text for approval. If the text is edited after approval, the approval no longer matches the payload, so the application requests approval again. Once the note is submitted, the workflow reads back the case state or note identifier from the system of record to verify the outcome.

The implementation sequence follows the boundaries rather than the feature list. First, test that the application passes the correct caller context. Second, test valid and denied case lookups directly at the service boundary. Third, connect the workflow to the approved read operations and verify that a request for a different customer’s case is rejected. Fourth, add the draft-only response. Finally, introduce the write operation behind the explicit approval and result-verification steps.

An illustrative capacity calculation can help identify which team needs to estimate demand. Assume, solely for planning, 150 active tasks during a busy interval, with each task making an average of two tool calls per minute. That implies about 300 tool requests per minute at the assumed average, before bursts, retries, or uneven traffic. It is not a Foundry quota or a performance prediction. The value of the calculation is that it directs the team to measure the actual case service, connection handling, and queue behavior under its own expected request pattern.

This sequence preserves a useful distinction in the product decision. A draft is a recommendation that a person can inspect. A submitted change is a business action whose exact payload, authorization, and outcome need to be handled by the appropriate application and system boundaries. The team can add more automation later, but each additional operation should have a named owner and a release test rather than inheriting broad authority from the original summarization use case.

8. Plan for failure across component boundaries

An agent workflow can fail even when each component appears healthy in isolation. A model response may be available while a tool is unreachable. A tool may respond while its data is stale. A write request may time out after the downstream system has accepted it, leaving the caller uncertain whether the action occurred. Design the workflow to represent these states explicitly instead of collapsing them into a generic success or failure message.

Consider a failure timeline for the hypothetical case update. The application submits an approved note request. The tool gateway forwards it to the case system. The connection times out before the application receives a response. The application must not assume that the action failed and blindly submit the same change again. It should report an unresolved outcome, check the case system for evidence of the requested update, and route the case for reconciliation if it cannot establish the result. This is a design recommendation; do not promise exactly-once effects from an uncertain network exchange.

Use a small set of operational states that distinguish what the user and support team need to know: request rejected before execution, request accepted for processing, result confirmed, result denied by the owning system, and outcome unknown pending verification. Map each state to an owner and next action. For example, an authorization denial belongs with the system or data owner to explain the policy decision; an unavailable route belongs with the network or platform owner; an unknown write outcome belongs with the service owner responsible for reconciliation.

For each dependency, define a timeout response and decide whether a retry is safe. A read may often be repeated without changing business state, though repeated reads still have capacity and cost implications for the system. A write should only be retried under a contract that lets the receiving system identify or reconcile a repeated request. If the receiving system cannot provide that behavior, the application should favor verification and human resolution over an automatic retry that could duplicate an action.

Keep records sufficient to trace a request across the application, runtime, tool interface, and target system. A correlation identifier can help teams locate related events if the implementation supports passing and recording one consistently. Define what information is safe and necessary to log; avoid copying full sensitive prompts or returned records into logs merely to make troubleshooting easier. The service owner should be able to locate an event without turning diagnostic storage into another uncontrolled data repository.

Run failure exercises before release. Make a tool unavailable and check that the workflow reports a bounded failure rather than inventing a result. Deny a valid-looking operation at the owning system and confirm that the application does not present it as completed. Simulate an uncertain response after a write and exercise the reconciliation path. These tests are meaningful only when the expected outcome and responsible team are recorded in advance.

9. Use the architecture worksheet as a release artifact

The worksheet below is intended to be copied into a design record and completed by the people who own the boundaries. It is not a downloadable template or a substitute for reviewing the implementation. Its value is to make unanswered questions visible while a change is still inexpensive to adjust.

Complete one worksheet per distinct user journey when the paths or business rights differ materially. Two workflows may share a runtime and still need separate tool contracts because one reads customer history while another changes service status. Conversely, do not create duplicate records solely because the screen or prompt differs if the underlying call paths, rights, and owners are genuinely the same.

A practical design review should be able to trace one allowed request from entry to result, then trace one denied request to the component that blocks it. It should also explain what the user sees when a dependency is unavailable and who investigates an unresolved action. If the worksheet cannot answer those points, the architecture is not ready for a release decision.

For a compact review, ask owners to bring evidence rather than assurances. The application owner can demonstrate that caller context is established as intended. The data owner can show a valid access case and a cross-tenant denial. The tool owner can show the operation contract and rejected inputs. The network owner can confirm the required paths. The service owner can walk through the failure and reconciliation procedure. The evidence can be appropriate to the organization’s release process; it does not need to be a single universal test format.

Printable boundary review

Use this grouped checklist during a design or release meeting. The explanations are included so the sheet remains useful when separated from the surrounding article.

Architecture and ownership

• The user journey is specific. Name the input, the information returned, and any business action that may follow; avoid a broad label such as “agent support.”
• Each boundary has an accountable owner. Assign the runtime, tool, data, network, and business-system decisions, even if one team owns several of them.
• Every arrow has a purpose. Record the caller, destination, information passed, access decision, and failure owner for each required path.
• The chosen runtime approach fits the operating model. Document who builds, deploys, supports, and changes it without assuming that the deployment choice supplies the organization’s access policy.

Data and tools

• The data service checks access. Test a valid lookup and a nearby unauthorized lookup at the service that returns the records.
• Tool operations are bounded. List each permitted operation and its inputs rather than describing access as a broad connection to an enterprise system.
• Write actions have a distinct path. Identify where the action is authorized, how an approval binds to the exact payload, and how the resulting business state is verified.
• Tool changes have an owner. Define who reviews versions, permissions, and contract changes before a workflow depends on them.

Identity, network, and operations

• Caller context is distinguishable from user text. State which values come from a trusted application path and which are merely requested inputs.
• Inbound and outbound paths are both reviewed. Confirm required destinations and tool compatibility for the selected configuration.
• Failure states are distinct. Decide how rejection, dependency failure, and an uncertain write outcome are reported and investigated.
• Release evidence is retained. Keep the call-path diagram, decision owners, test results, and runbook with the deployment record.

Printable summary: Journey and caller context; runtime and model path; tool operations and owners; data and action authorization points; inbound and outbound routes; one valid test; one denied test; one failure exercise; result verification; operational owner.

10. Sequence implementation and next steps

Build the architecture in an order that exposes boundary errors before the workflow becomes difficult to change. Start with a single user journey and a read-only operation. Confirm the application’s caller context and the data service’s access check before adding more tools or actions. This is not a claim that read-only systems have no risk; it is a way to validate the core separation without beginning with a state-changing operation.

Next, formalize the tool contract and the owners who will maintain it. Keep the first operation set aligned to the demonstrated need. When a second operation is requested, review whether it requires a different permission, data owner, route, or failure response. Add it as a deliberate extension, not as an incidental consequence of expanding a general-purpose tool.

Then test the end-to-end journey across team boundaries. Include a successful request, a request that should be denied, a dependency failure, and an unresolved action outcome if the workflow can write business data. Ensure the teams can tell which component produced each result and know who will investigate it. Use those tests to update the diagram and responsibility matrix rather than keeping implementation details in engineers’ memory.

Finally, set a change trigger. A new tool, data source, model endpoint, network route, or business action should prompt review of the paths and owners it affects. The review need not reopen every settled platform decision. It should confirm that the new component has not created a route around an existing data check or introduced an operation without an accountable owner.

If your organization needs help establishing these architecture reviews, ownership records, and engineering practices across teams, explore enterprise platform engineering. The immediate next step is to choose one representative user journey and complete the worksheet with the application, platform, tool, data, network, and service owners. A sound design is one those teams can explain together: what the agent can call, which system decides access, and what happens when the intended path does not work.

WANT TO TALK THROUGH YOUR SITUATION?

Talk it through with us. Thirty minutes, no slides.