APRA CPS 234 and Third-Party AI: What Trustees Demand

Table of Contents

1. Why APRA CPS 234 Matters for Third-Party AI
2. What APRA CPS 234 Actually Requires
3. Third-Party AI Risk and Control Landscape
4. Governance and Board Accountability
5. Technical Controls and Evidence Patterns
6. Vendor Management and Due Diligence
7. Audit Readiness and Documentation
8. Implementation Roadmap
9. Common Pitfalls and How to Avoid Them
10. Next Steps: Getting Started

---

Why APRA CPS 234 Matters for Third-Party AI

If you’re a trustee, board member, or chief risk officer at an Australian financial services firm—bank, wealth manager, fund manager, superannuation fund, or insurer—you’ve likely heard that APRA CPS 234 is non-negotiable. But what many boards don’t yet realise is that the moment you deploy third-party AI, your compliance obligations don’t shrink. They expand.

APRA Prudential Standard CPS 234: Information Security is Australia’s baseline information security standard for APRA-regulated entities. It covers governance, risk management, incident response, and testing. It’s technology-agnostic. But when you layer third-party AI—whether that’s a large language model (LLM) from OpenAI or Claude, a bespoke ML model from a fintech vendor, or an agentic workflow from a SaaS platform—you inherit new risks that CPS 234 explicitly requires you to manage, control, and evidence.

The problem: most trustees and boards treat third-party AI as a “vendor risk” problem. They tick a box in a vendor questionnaire, get a SOC 2 report, and move on. That’s insufficient. APRA doesn’t just want to know that your vendor has good security. It wants to know that you understand what the AI does, what could go wrong, how you’ll detect it, and how you’ll respond. And it wants evidence.

This guide walks through what APRA actually expects, how to structure controls around third-party AI, what audit evidence looks like in practice, and the exact implementation steps we use on customer engagements at PADISO’s AI for Financial Services Sydney practice.

---

What APRA CPS 234 Actually Requires

The Four Pillars of CPS 234

APRA CPS 234 is built on four core pillars:

1. Information Security Capability — Your organisation must have the people, processes, and technology to manage information security risks.
2. Controls — You must design, implement, and test controls that protect confidentiality, integrity, and availability of information.
3. Testing and Assurance — You must independently test controls at least annually and report results to the board.
4. Incident Notification — You must detect, respond to, and notify APRA of material security incidents within defined timeframes.

For third-party AI, each pillar has a distinct set of expectations.

Capability Expectation: Do You Understand Your AI?

APRA expects your organisation to have demonstrable capability in understanding and managing third-party AI risks. This means:

• AI Risk Inventory — A documented list of every third-party AI system in use, what it does, where it touches customer data, and what could go wrong.
• Risk Assessment — For each system, a formal risk assessment that identifies confidentiality, integrity, and availability risks.
• Governance — Clear ownership (usually CISO + Chief Data Officer or equivalent) and a defined approval process for new AI systems.
• Competency — At least one person on your technology or security team who understands machine learning risks, prompt injection, model drift, and hallucination.

Many organisations fail here. They deploy an AI tool because it saves time or money, but no one on the security team has been trained to assess what could go wrong. APRA sees this and marks it down.

Controls Expectation: What Must You Actually Do?

APRA’s control expectations for third-party AI fall into five categories:

1. Access and Data Controls

• Who can use the AI system? (Role-based access control.)
• What data can they feed it? (Data classification and masking.)
• Can they export or copy outputs? (Audit logging.)

2. Model and Prompt Controls

• Is the model version pinned and documented? (Reproducibility.)
• Are prompts version-controlled? (Auditability.)
• Are there guardrails or jailbreak mitigations? (Risk reduction.)

3. Output Validation and Human Oversight

• Does a human review AI outputs before they’re used in a decision? (Particularly for lending, claims, or conduct-risk decisions.)
• Is there a process to flag or escalate suspicious outputs? (Anomaly detection.)
• Are outputs logged and traceable? (Auditability.)

4. Vendor and Third-Party Controls

• Does your vendor have security certifications (SOC 2, ISO 27001)? (Hygiene.)
• Can your vendor access your data? If so, under what contract terms and monitoring? (Data residency and access logs.)
• What’s the vendor’s incident response plan and your notification SLA? (Incident readiness.)

5. Incident and Change Controls

• How will you detect if the AI model has been poisoned, drifted, or manipulated? (Monitoring.)
• What’s your incident response process if the AI produces harmful outputs? (Escalation.)
• How do you version, test, and roll back AI model updates? (Change management.)

These aren’t optional. They’re the baseline APRA expects to see in your control documentation.

---

Third-Party AI Risk and Control Landscape

The Unique Risks of Third-Party AI

Third-party AI introduces risks that traditional software doesn’t. Understanding them is the first step to controlling them.

Model Drift and Degradation Third-party models (especially LLMs) are often retrained or fine-tuned by the vendor without your knowledge. If a model’s performance degrades—accuracy drops, hallucinations increase, bias emerges—you may not notice until it affects customer outcomes or regulatory metrics. APRA expects you to have monitoring in place to detect this.

Prompt Injection and Adversarial Attacks If your staff can input custom prompts into a third-party LLM, a malicious actor (or an untrained employee) could craft a prompt that makes the model ignore its guardrails, leak training data, or produce biased outputs. OWASP Machine Learning Security Top 10 documents these attack surfaces. APRA expects you to have mitigations.

Data Leakage If you feed confidential customer data into a third-party LLM (e.g., OpenAI’s API), that data may be retained by the vendor, used to train future models, or exposed in a breach. Even if the vendor’s SOC 2 report says “no data retention,” APRA expects you to have a contractual commitment and monitoring to verify it.

Vendor Lock-In and Continuity If your AI vendor goes out of business, gets acquired, or changes their terms, you may lose access to a critical system or be forced to migrate at short notice. APRA expects you to have a continuity plan.

Hallucination and Accuracy LLMs hallucinate—they generate plausible-sounding but false information. If an LLM is used to summarise customer complaints, generate compliance reports, or assist in lending decisions, hallucinations can directly harm customers or breach regulatory obligations. APRA expects you to have controls to detect and mitigate this.

Bias and Fairness Third-party models may encode historical biases in training data. If an AI system is used in lending, insurance underwriting, or customer segmentation, biased outputs can breach anti-discrimination laws and APRA’s Conduct Risk expectations. APRA expects you to have testing and monitoring in place.

Mapping Risks to Controls

For each risk category, there’s a corresponding control pattern:

Risk	Control Pattern	Evidence
Model drift	Continuous monitoring of model accuracy, latency, and output distribution	Monitoring dashboards, alert logs, escalation records
Prompt injection	Input validation, prompt templates, guardrails, user training	Code review, security test results, training records
Data leakage	Data classification, masking, contractual terms, access logs	Data flow diagram, contract excerpt, vendor attestation, logs
Vendor continuity	Vendor due diligence, continuity plan, backup system	Due diligence checklist, continuity plan document, test results
Hallucination	Human review, output validation, anomaly detection	Review logs, validation rules, alert logs
Bias	Training data audit, fairness testing, monitoring	Fairness test report, monitoring dashboards, incident logs

This table is your control framework. Every third-party AI system should map to at least three of these patterns.

---

Governance and Board Accountability

What the Board Needs to Know

APRA expects boards to have explicit oversight of AI risk. This doesn’t mean board members need to understand machine learning. It means they need to understand:

1. What third-party AI systems are in use? (Inventory.)
2. What could go wrong with each? (Risk assessment.)
3. How are we controlling the risks? (Control summary.)
4. How do we know the controls are working? (Testing and monitoring.)
5. What’s our incident response plan? (Escalation and notification.)

We recommend a quarterly “AI Risk Dashboard” that the board reviews. It should include:

• AI System Inventory — Name, vendor, use case, data sensitivity, risk rating.
• Control Status — For each system, a green/amber/red status on the five control categories above.
• Testing Results — Results of the most recent security test, fairness audit, or accuracy validation.
• Incidents and Near-Misses — Any security incidents, hallucinations, or unexpected outputs in the quarter.
• Vendor Status — Any changes to vendor terms, security certifications, or continuity status.

This dashboard is not just governance theatre. It’s evidence that the board is actively managing AI risk. APRA examiners will ask for it.

Defining Roles and Accountability

For third-party AI governance, you need clear role definitions:

Chief Information Security Officer (CISO)

• Overall accountability for third-party AI security controls.
• Approval authority for new third-party AI systems.
• Incident response escalation point.

Chief Data Officer (CDO) or Head of Analytics

• Accountability for data governance and masking.
• Model accuracy and fairness monitoring.
• Vendor data-handling verification.

Chief Risk Officer (CRO)

• Accountability for third-party AI risk assessment and rating.
• Conduct risk and fairness monitoring.
• Board reporting.

Business Unit Owner

• Day-to-day management of the AI system.
• Prompt management and output review (if applicable).
• Incident detection and escalation.

Without clear role definitions, accountability diffuses and controls fail. APRA expects to see this documented.

---

Technical Controls and Evidence Patterns

Access and Authentication

What APRA Expects

• Role-based access control (RBAC) to AI systems.
• Multi-factor authentication (MFA) for sensitive systems.
• Audit logging of all access and actions.
• Regular access reviews (quarterly minimum).

Implementation Pattern

Access Request → Approval Workflow → RBAC Assignment → Audit Log

Evidence You’ll Need

• Access control matrix (who can access what).
• Approval records for the past 12 months.
• Audit logs showing access, actions, and timestamps.
• Quarterly access review sign-offs.

For third-party AI systems (especially cloud-based LLMs), ensure your identity and access management (IAM) system can integrate with the vendor’s API. If it can’t, you can’t audit who accessed the system. That’s a control failure.

Data Classification and Masking

What APRA Expects

• A data classification policy that defines what data can be fed into third-party AI systems.
• Masking or anonymisation of sensitive data (PII, customer account numbers, etc.) before it’s sent to a third-party AI.
• Documentation of the masking rules and their effectiveness.

Implementation Pattern

Raw Data → Classification → Masking Rules → Masked Data → Third-Party AI

Evidence You’ll Need

• Data classification policy and approval.
• Masking rule documentation and test results.
• Sample data flows showing before/after masking.
• Vendor attestation that masked data is not retained or used for training.

This is a critical control. If you’re feeding customer names, account numbers, or transaction details into an LLM without masking, you’re violating CPS 234 and creating a data breach risk. APRA will flag this immediately.

Model and Prompt Version Control

What APRA Expects

• Documented version of the AI model in use (e.g., GPT-4, Claude 3.5 Sonnet, custom model v2.1).
• Version control for any custom prompts or instructions.
• Traceability between outputs and the model/prompt version used to generate them.
• A change management process for model or prompt updates.

Implementation Pattern

Prompt Repository → Version Control → Testing → Approval → Deployment → Audit Trail

Evidence You’ll Need

• Prompt version control repository (GitHub, GitLab, etc.) showing commit history.
• Testing results for each prompt or model version change.
• Change approval records.
• Output audit logs that include model and prompt version.

If you can’t trace an output back to the exact model and prompt that generated it, you can’t defend the output if it’s wrong or biased. APRA will ask for this traceability.

Output Validation and Human Oversight

What APRA Expects

• For high-risk decisions (lending, claims, conduct-risk flagging), a human must review and approve AI outputs before they’re used.
• A documented validation process that checks outputs for accuracy, bias, and hallucination.
• Escalation procedures for suspicious or anomalous outputs.
• Audit logging of validation decisions.

Implementation Pattern

AI Output → Validation Rules → Human Review → Approval/Rejection → Audit Log

Evidence You’ll Need

• Validation rules or checklist.
• Sample of reviewed outputs (anonymised) showing human sign-off.
• Escalation logs showing how many outputs were flagged and why.
• Training records for staff who perform validation.

This is where many organisations fail. They deploy an AI system and assume it’s accurate. APRA expects you to prove that humans are actually reviewing outputs and catching errors. If you can’t show validation evidence, you’re not compliant.

Vendor Security and Data Handling

What APRA Expects

• Vendor security certifications (SOC 2 Type II, ISO 27001).
• Written confirmation that the vendor does not retain, train on, or share your data.
• Contractual terms that allow you to audit the vendor’s security controls.
• A vendor incident notification SLA (usually 24-48 hours for material incidents).
• Evidence that you’ve verified the vendor’s claims (e.g., through a security questionnaire or audit).

Implementation Pattern

Vendor Selection → Due Diligence → Contract Negotiation → Attestation → Monitoring

Evidence You’ll Need

• Vendor due diligence questionnaire and responses.
• SOC 2 report or ISO 27001 certificate.
• Data Processing Addendum (DPA) or equivalent.
• Vendor attestation on data retention and use.
• Quarterly vendor status checks.

For cloud-based LLMs like OpenAI, many organisations assume the vendor is “secure enough” because they’re well-known. But APRA expects you to have formal verification. We recommend getting a signed attestation from the vendor confirming data handling, running a security questionnaire, and reviewing their latest SOC 2 report.

Monitoring and Anomaly Detection

What APRA Expects

• Continuous or regular monitoring of AI system performance, accuracy, and output distribution.
• Alerts for anomalies (e.g., accuracy drop, unusual output patterns, excessive hallucinations).
• Documented response procedures for alerts.
• Trending and root-cause analysis of anomalies.

Implementation Pattern

AI System → Metrics Collection → Threshold Monitoring → Alert → Investigation → Resolution

Evidence You’ll Need

• Monitoring dashboard or logs showing key metrics (accuracy, latency, output distribution).
• Alert threshold definitions and justification.
• Alert logs from the past 12 months.
• Investigation and resolution records for each alert.

For example, if you’re using an AI system to flag suspicious transactions, you’d monitor:

• True positive rate — What percentage of flagged transactions are actually suspicious?
• False positive rate — What percentage are false alarms?
• Alert volume — Is the system generating more or fewer alerts than expected?
• Response time — How quickly are alerts being reviewed and acted on?

If accuracy drops by 10% in a quarter, that’s an anomaly. APRA expects you to investigate why and take corrective action.

---

Vendor Management and Due Diligence

The Third-Party AI Due Diligence Checklist

Before you contract with a third-party AI vendor, you need a formal due diligence process. Here’s what we use with our clients at PADISO’s AI Advisory Services Sydney:

Organisational and Financial

• Vendor company registration and financial stability (credit check, revenue, funding).
• Vendor ownership and change of control history.
• Vendor’s own regulatory compliance (if applicable).
• Insurance (E&O, cyber liability).

Security and Compliance

• SOC 2 Type II report (or equivalent).
• ISO 27001 certification (or equivalent).
• Vulnerability disclosure policy.
• Incident response plan and notification SLA.
• Data protection and privacy certifications (GDPR, CCPA, etc.).

Data Handling

• Written confirmation: Does the vendor retain your data? For how long?
• Written confirmation: Does the vendor use your data to train or improve models?
• Written confirmation: Does the vendor share your data with third parties?
• Data residency and geographic restrictions.
• Data deletion and retention policies.

Model and AI Governance

• Model training data sources and composition.
• Fairness and bias testing results.
• Model update frequency and notification process.
• Hallucination rates and mitigation strategies.
• Model explainability and interpretability.

Continuity and Exit

• Vendor’s disaster recovery and business continuity plan.
• Data export capabilities (format, timeline).
• Model export or licensing terms (if applicable).
• Contract termination and wind-down procedures.

Contractual and Legal

• Data Processing Addendum (DPA) or equivalent.
• Liability and indemnification terms.
• Audit rights (can you audit the vendor’s controls?).
• Subcontractor management (who else has access to your data?).
• Dispute resolution and governing law.

This checklist is not exhaustive, but it covers the critical areas. For each item, document the vendor’s response and your assessment. If the vendor won’t answer a question, that’s a red flag.

Ongoing Vendor Monitoring

Due diligence doesn’t end at contract signature. APRA expects continuous vendor monitoring:

Quarterly

• Review vendor status (any security incidents, breaches, or service disruptions?).
• Check for SOC 2 or ISO 27001 certificate renewal or updates.
• Review vendor’s public security advisories or incident disclosures.

Annually

• Request updated SOC 2 report or security questionnaire.
• Review vendor’s model update history and release notes.
• Conduct a fairness or bias audit on the vendor’s model.
• Review vendor’s incident response SLA compliance.

As Needed

• If the vendor changes ownership, undergoes a major update, or has a public security incident, trigger an urgent review.

Document all monitoring activities. When APRA asks, “How do you monitor your third-party AI vendors?” you’ll have evidence.

---

Audit Readiness and Documentation

What APRA Examiners Will Ask

When APRA conducts a compliance examination (which they do regularly), they’ll focus on third-party AI with these questions:

1. Inventory — “Show us every third-party AI system in use. How many are there? What do they do?”
2. Risk Assessment — “For each system, show us the risk assessment. What could go wrong?”
3. Controls — “What controls do you have in place? Show us the documentation.”
4. Testing — “How do you test that controls are working? Show us the results.”
5. Incidents — “Have there been any security incidents, data breaches, or AI-related failures? How did you respond?”
6. Vendor Management — “How do you manage vendor risk? Show us the due diligence and monitoring.”
7. Board Oversight — “Does the board know about third-party AI risks? Show us board papers or dashboards.”

If you can’t answer these questions with evidence, you’re not compliant. Period.

Building Your Evidence Package

We recommend organising your evidence into a “Third-Party AI Compliance Folder” with these sections:

1. AI System Inventory

• Spreadsheet listing every third-party AI system.
• For each: name, vendor, use case, launch date, data sensitivity, risk rating.
• Update quarterly.

2. Risk Assessments

• One risk assessment document per third-party AI system.
• Format: [System Name] Risk Assessment, dated, signed by CISO and CRO.
• Content: What the system does, what data it touches, what could go wrong, inherent risk rating, controls, residual risk rating.

3. Control Documentation

• Access control matrix (who can access what).
• Data classification and masking policy.
• Prompt version control repository (or screenshots).
• Output validation procedures and sample logs.
• Vendor due diligence questionnaire and responses.
• Vendor SOC 2 reports and attestations.
• Monitoring dashboards and alert logs.

4. Testing and Assurance

• Annual security testing results (penetration tests, code review, etc.).
• Fairness and bias audit results.
• Accuracy and performance testing results.
• Access control testing (verify RBAC is working).
• Data masking testing (verify masked data is effective).

5. Incidents and Near-Misses

• Log of any security incidents, data breaches, or AI-related failures.
• For each: date, description, impact, response, resolution.
• Root-cause analysis and corrective actions.

6. Board and Governance

• Board papers or dashboards on third-party AI risk (quarterly).
• Board meeting minutes discussing AI risk.
• Policy documents (AI risk policy, vendor management policy, etc.).
• Role and responsibility definitions.

7. Vendor Monitoring

• Quarterly vendor status check records.
• Updated SOC 2 reports or security questionnaires.
• Vendor incident notifications (if any).
• Model update logs and release notes.

Keep this folder up to date. When APRA calls for an examination, you’ll be ready.

Documentation Standards

APRA expects documentation to be:

• Dated and Signed — Who approved this? When?
• Versioned — What version is this? When was it last updated?
• Specific — No generic statements. “We have strong controls” is not evidence. “We have RBAC implemented via Azure AD, verified by quarterly access reviews” is evidence.
• Auditable — Can an external auditor verify the claim? If you say “we monitor accuracy,” can you show the monitoring dashboard and alert logs?

---

Implementation Roadmap

Phase 1: Inventory and Assessment (Weeks 1-4)

Week 1: AI System Inventory

• Conduct a sweep of your organisation to identify all third-party AI systems in use.
• Include: ChatGPT, Copilot, vendor-provided AI, custom ML models, RPA tools with AI, etc.
• Create a simple spreadsheet: System Name, Vendor, Use Case, Data Sensitivity, Owner.
• Target: 30-50 systems for most mid-market financial services firms.

Week 2: Risk Assessment

• For each system, conduct a quick risk assessment.
• Use a template: What does it do? What data does it touch? What could go wrong? Risk rating (high/medium/low).
• Prioritise high-risk systems (those touching customer data or used in lending/claims decisions).

Week 3: Control Mapping

• For each high-risk system, map the five control categories (access, data, model, output, vendor).
• Document current state: Do you have this control? Is it documented?
• Identify gaps.

Week 4: Board and Governance

• Present inventory and risk assessment to the board or risk committee.
• Define roles and accountability (CISO, CDO, CRO, business unit owners).
• Approve a remediation roadmap.

Deliverables

• AI System Inventory (spreadsheet).
• Risk Assessment Summary (one page per high-risk system).
• Control Gap Analysis (spreadsheet).
• Board Paper (2-3 pages).

Phase 2: Control Build-Out (Weeks 5-12)

Weeks 5-6: Access and Data Controls

• Implement RBAC for AI systems (integrate with IAM if possible).
• Define data classification policy and masking rules.
• Test masking effectiveness on sample data.
• Document access control matrix.

Weeks 7-8: Model and Vendor Controls

• Version-control all prompts and custom instructions.
• Conduct vendor due diligence (questionnaire, SOC 2 review, attestation).
• Negotiate or update contracts to include data handling terms and audit rights.
• Document vendor risk ratings.

Weeks 9-10: Output Validation and Monitoring

• Define output validation procedures for high-risk systems.
• Set up monitoring dashboards (accuracy, latency, alert volume).
• Define alert thresholds and escalation procedures.
• Train staff on validation and escalation.

Weeks 11-12: Testing and Documentation

• Conduct security testing (code review, vulnerability scan if applicable).
• Conduct fairness or bias audit.
• Conduct access control testing.
• Document all results.

Deliverables

• Access Control Matrix (with approval records).
• Data Classification and Masking Policy (approved).
• Vendor Due Diligence Package (for each vendor).
• Output Validation Procedures (documented).
• Monitoring Dashboard (live).
• Testing Results (security, fairness, accuracy).

Phase 3: Governance and Monitoring (Weeks 13-16)

Week 13: Board Dashboard

• Create quarterly AI Risk Dashboard (system inventory, control status, testing results, incidents).
• Present to board or risk committee.
• Establish quarterly reporting cadence.

Week 14: Incident Response

• Define incident response procedures for AI-related incidents (data breach, hallucination, model drift, vendor incident).
• Assign incident response team members and escalation paths.
• Run a tabletop exercise.

Week 15: Vendor Monitoring

• Establish quarterly vendor monitoring process (status check, SOC 2 review, incident check).
• Document vendor risk ratings and monitoring results.

Week 16: Policy and Training

• Finalise AI Risk Policy, Vendor Management Policy, and Incident Response Policy.
• Train staff (board, CISO team, business unit owners, end users).
• Document training records.

Deliverables

• Quarterly AI Risk Dashboard (board-ready).
• Incident Response Plan (documented and tested).
• Vendor Monitoring Process (documented).
• AI Risk Policy (approved by board).
• Training Records (staff, board, end users).

Post-Implementation: Continuous Improvement

• Monthly — Monitor alerts and anomalies. Investigate and resolve.
• Quarterly — Update AI Risk Dashboard. Review with board. Monitor vendors.
• Annually — Conduct security testing, fairness audit, and control effectiveness review. Update policies.
• As Needed — Respond to incidents, vendor changes, or regulatory updates.

This is not a one-time project. It’s an ongoing program. APRA expects continuous improvement.

---

Common Pitfalls and How to Avoid Them

Pitfall 1: Treating Third-Party AI as a Vendor Risk Checkbox

The Problem You get a SOC 2 report from your vendor and assume you’re compliant. You don’t actually understand what the AI does or what could go wrong.

The Fix

• Conduct your own risk assessment, independent of the vendor’s security posture.
• Understand the specific AI risks (model drift, hallucination, bias, data leakage).
• Implement controls that are specific to those risks, not generic vendor controls.
• Document your understanding and controls. That’s what APRA wants to see.

Pitfall 2: No Human Oversight of AI Outputs

The Problem You deploy an AI system and assume it’s accurate. Staff use AI outputs directly in decisions without reviewing them.

The Fix

• For any AI system used in a material decision (lending, claims, conduct-risk flagging), require human review and approval.
• Document the validation process and sample review logs.
• Train staff on what to look for (accuracy, bias, hallucinations).
• Track how many outputs are reviewed, how many are rejected, and why.

Pitfall 3: No Monitoring or Anomaly Detection

The Problem You deploy an AI system and don’t monitor it. If accuracy degrades or the model drifts, you don’t notice until a customer complains or an audit fails.

The Fix

• Set up continuous monitoring of key metrics (accuracy, latency, output distribution, alert volume).
• Define alert thresholds based on your risk appetite.
• Create an escalation procedure for alerts.
• Investigate and document the root cause of every anomaly.

Pitfall 4: Feeding Sensitive Data into Third-Party AI Without Masking

The Problem You feed customer names, account numbers, or transaction details into an LLM without masking. The vendor retains the data or uses it to train models. You’ve just created a data breach and compliance violation.

The Fix

• Classify your data and define what can be fed into third-party AI.
• Implement masking or anonymisation for sensitive data.
• Test masking effectiveness.
• Get vendor attestation that data is not retained or used for training.
• Audit vendor compliance quarterly.

Pitfall 5: No Version Control for Prompts or Models

The Problem You update a prompt or the vendor updates their model. You can’t trace an output back to the exact version that generated it. If the output is wrong, you can’t investigate or defend it.

The Fix

• Version-control all custom prompts (GitHub, GitLab, etc.).
• Document the model version in use (e.g., GPT-4, Claude 3.5 Sonnet).
• Include model and prompt version in output audit logs.
• Implement a change management process for updates.

Pitfall 6: Board Unaware of Third-Party AI Risks

The Problem The board doesn’t know what third-party AI systems are in use or what could go wrong. When APRA asks, you can’t show board oversight.

The Fix

• Create a quarterly AI Risk Dashboard for the board.
• Include: system inventory, control status, testing results, incidents, vendor status.
• Present and discuss at board or risk committee meetings.
• Document board discussions in meeting minutes.

Pitfall 7: Vendor Due Diligence is Superficial

The Problem You ask the vendor a few questions, they say “we’re secure,” and you move on. You don’t actually verify their claims or understand their controls.

The Fix

• Use a comprehensive due diligence questionnaire (we’ve provided a checklist above).
• Review the vendor’s SOC 2 report or ISO 27001 certificate.
• Get written attestations on data handling, retention, and use.
• For critical vendors, conduct a security audit or require a third-party assessment.
• Repeat annually.

---

Next Steps: Getting Started

If you’re responsible for AI risk at an Australian financial services firm, here’s what to do next:

Immediate (This Week)

1. Inventory — Conduct a quick sweep of your organisation. List every third-party AI system in use. If you can’t list them, you have a governance problem.
2. Escalate — Bring this to your CISO, CRO, or board. Third-party AI risk is a material compliance issue. It needs executive attention.
3. Assign Ownership — Designate someone (CISO, CRO, or equivalent) to own third-party AI risk and compliance. Without an owner, nothing will happen.

Short Term (Next 4 Weeks)

1. Risk Assessment — For your top 5-10 third-party AI systems, conduct a formal risk assessment. Document what could go wrong and what controls you need.
2. Control Gap Analysis — For each high-risk system, map the five control categories (access, data, model, output, vendor). Identify gaps.
3. Board Paper — Prepare a 2-3 page board paper summarising your inventory, risks, and remediation plan. Get board approval.

Medium Term (Weeks 5-12)

1. Control Build-Out — Implement the controls identified in your gap analysis. Prioritise high-risk systems.
2. Vendor Due Diligence — For each critical vendor, conduct formal due diligence. Get SOC 2 reports, data handling attestations, and audit rights.
3. Testing and Documentation — Conduct security testing, fairness audits, and access control testing. Document results.

Ongoing

1. Quarterly Board Dashboard — Create and present a quarterly AI Risk Dashboard to the board.
2. Quarterly Vendor Monitoring — Monitor vendor security, incidents, and model updates.
3. Annual Testing — Conduct annual security testing and fairness audits. Update policies and training.

If You Need Help

If this feels overwhelming, you’re not alone. Many Australian financial services firms are just starting to grapple with third-party AI compliance. At PADISO, we work with banks, wealth managers, funds, and insurers to build third-party AI governance from the ground up.

Our approach:

1. AI Quickstart Audit — In 2 weeks, we assess your current state, identify risks, and recommend priorities. We give you a clear roadmap. Book a fixed-fee audit here.
2. AI Strategy & Readiness — We help you define your AI risk policy, governance model, and control framework. We work with your board and executive team.
3. Fractional CTO Leadership — If you need ongoing technical leadership on AI and security, we provide fractional CTO advisory tailored to financial services.
4. Security Audit & Vanta Implementation — If you’re also pursuing SOC 2 or ISO 27001 compliance, we can help you get audit-ready in weeks, not months, using Vanta to automate evidence collection.

Our team is based in Sydney and has deep experience with APRA, ASIC, and AUSTRAC compliance. We speak the language of trustees and boards. We lead with concrete results, not hype. And we work at your pace—whether that’s a quick audit, a 12-week build-out, or ongoing advisory.

Start with a 30-minute call to discuss your situation. We’ll help you understand where you stand and what comes next.

---

Summary

APRA CPS 234 compliance for third-party AI is not optional. It’s a material obligation for every APRA-regulated entity using third-party AI systems. The standard is technology-agnostic, but the risks are real: model drift, data leakage, hallucination, bias, vendor lock-in, and continuity.

APRA expects you to:

1. Understand what third-party AI systems you’re using and what could go wrong.
2. Control the risks through governance, access controls, data masking, output validation, vendor management, and monitoring.
3. Test your controls at least annually and report results to the board.
4. Respond to incidents and notify APRA when required.
5. Evidence all of the above with documentation that an external auditor can verify.

This guide has walked through the practical steps to achieve this. The roadmap is clear: inventory (4 weeks), control build-out (8 weeks), governance and monitoring (4 weeks), and then continuous improvement.

If you’re just starting, begin with an inventory and risk assessment. If you’re further along, focus on evidence and board oversight. Either way, the time to act is now. APRA is actively examining third-party AI risk, and examiners are looking for exactly what we’ve outlined here.

You have the playbook. Now execute it.