Apache Superset on ECS Fargate: Reference Deployment Pattern

Table of Contents

1. Why Superset on Fargate
2. Architecture Overview
3. Pre-deployment Prerequisites
4. Networking and Load Balancing
5. Secrets and Environment Configuration
6. Database and Persistent Storage
7. Task Definition and Container Configuration
8. Service Deployment and Autoscaling
9. Observability, Logging, and Monitoring
10. Operational Runbooks
11. Cost Optimisation
12. Next Steps

---

Why Superset on Fargate

Apache Superset is a lightweight, open-source business intelligence platform that delivers embedded analytics, dashboards, and SQL exploration without the overhead of traditional BI tools. When you deploy it on AWS ECS Fargate, you get serverless container compute—no EC2 instances to patch, no capacity planning, and billing tied directly to the CPU and memory your containers actually consume.

For teams modernising analytics infrastructure, Superset on Fargate solves a real problem: you need a BI layer that scales with your data volume and user load, but you don’t want to operate Kubernetes or maintain a fleet of servers. Fargate abstracts that away. You define task resources, set autoscaling policies, and let AWS handle the underlying infrastructure.

This matters especially for platform engineering teams across Australia building multi-tenant SaaS platforms or data-heavy applications. Whether you’re in Sydney, Melbourne, or Canberra, the pattern holds: Fargate lets you ship analytics infrastructure quickly, scale on demand, and keep operational overhead low.

The trade-off is that Fargate is not free—you pay per vCPU-hour and GB-hour—so sizing and autoscaling discipline matter. This guide covers the entire production deployment, from networking and secrets through autoscaling and operational runbooks, with the specific operational habits that keep Superset reliable at scale.

---

Architecture Overview

A production Superset deployment on Fargate typically includes:

Compute Layer: ECS Fargate tasks running the Superset application container, fronted by an Application Load Balancer (ALB) for traffic distribution and SSL termination.

Data Layer: A PostgreSQL RDS database for Superset metadata (dashboards, users, queries, data source definitions), separate from your data warehouse or operational databases. Superset queries run against your actual data sources—Snowflake, ClickHouse, BigQuery, Postgres, etc.—not the metadata store.

Storage: EFS (Elastic File System) for persisting Superset configuration, custom plugins, and logs across task restarts. Alternatively, S3 for logs and backups if EFS overhead is a concern.

Secrets Management: AWS Secrets Manager or Parameter Store to hold database credentials, API keys, and Superset configuration secrets, injected at task startup.

Networking: A VPC with public subnets for the ALB, private subnets for Fargate tasks and RDS, security groups enforcing least-privilege access, and optional VPC endpoints for S3 and Secrets Manager to avoid NAT gateway costs.

Autoscaling: ECS Service autoscaling policies based on CPU and memory utilisation, with CloudWatch alarms to alert on unhealthy task counts or high latency.

This architecture is stateless from the perspective of the Fargate task itself—the task is ephemeral, and all state lives in RDS and EFS. This means you can terminate a task at any time and a new one will pick up seamlessly.

---

Pre-deployment Prerequisites

Before you begin, ensure you have:

AWS Account and IAM Permissions: You need permissions to create VPCs, security groups, RDS instances, ECS clusters, ALBs, CloudWatch resources, and Secrets Manager entries. A policy with ecs:*, rds:*, ec2:*, elasticloadbalancing:*, secretsmanager:*, logs:*, and cloudwatch:* is a starting point; refine to least-privilege in production.

Docker Image: A Superset Docker image, either the official Apache Superset image from Docker Hub or a custom image built from the Superset repository with your plugins, themes, and configuration baked in. For most teams, starting with the official image and using environment variables and mounted config files is faster.

RDS Instance: A PostgreSQL 13+ database for Superset metadata. This must be in the same VPC as your Fargate tasks and have a security group allowing inbound traffic on port 5432 from the Fargate security group. Allocate at least 20 GB of storage; Superset metadata is lightweight, but query history and cached results add up.

Domain Name and SSL Certificate: A domain (e.g., superset.company.com) and an ACM certificate for HTTPS. The ALB will terminate SSL; traffic from ALB to Fargate tasks can be HTTP (internal).

VPC and Subnets: A VPC with at least two availability zones, each with a public subnet (for ALB) and a private subnet (for Fargate and RDS). This ensures high availability; if one AZ fails, the other absorbs traffic.

---

Networking and Load Balancing

Set up your VPC and security groups first. This is the foundation everything else sits on.

VPC and Subnet Layout

Create a VPC with CIDR block 10.0.0.0/16 (or your preferred range). Within it, create subnets:

• Public Subnet AZ-A: 10.0.1.0/24 – for ALB
• Public Subnet AZ-B: 10.0.2.0/24 – for ALB
• Private Subnet AZ-A: 10.0.10.0/24 – for Fargate and RDS
• Private Subnet AZ-B: 10.0.11.0/24 – for Fargate and RDS

Attach an Internet Gateway to the VPC. Create a route table for public subnets with a route 0.0.0.0/0 → Internet Gateway. For private subnets, create a NAT Gateway in a public subnet and route 0.0.0.0/0 → NAT Gateway. This allows Fargate tasks to reach external services (e.g., to pull the Docker image, fetch data from your warehouse) without being directly internet-facing.

Security Groups

Create three security groups:

ALB Security Group (sg-alb):

• Inbound: 80 (HTTP) and 443 (HTTPS) from 0.0.0.0/0
• Outbound: All traffic to Fargate security group

Fargate Security Group (sg-fargate):

• Inbound: 8088 (Superset default port) from ALB security group
• Inbound: 3306 or 5432 (if you run a local cache like Redis in the task) from itself
• Outbound: 5432 to RDS security group (for metadata database), 443 to 0.0.0.0/0 (for external data sources and ECR pulls)

RDS Security Group (sg-rds):

• Inbound: 5432 from Fargate security group
• Outbound: None needed (RDS is read-only from Fargate’s perspective)

Application Load Balancer

Create an ALB in the VPC, spanning both public subnets. Configure:

Listeners:

• HTTP (port 80) → redirect to HTTPS (port 443)
• HTTPS (port 443) with your ACM certificate → target group

Target Group:

• Protocol: HTTP
• Port: 8088 (Superset’s default)
• VPC: Your VPC
• Health check: Path /health, interval 30 seconds, healthy threshold 2, unhealthy threshold 3

The target group will be populated by your ECS service once it’s created. The health check ensures the ALB only sends traffic to healthy tasks.

---

Secrets and Environment Configuration

Superset requires a database URI, secret key, and optionally credentials for external data sources. Store these securely in AWS Secrets Manager.

Secrets Manager Setup

Create a secret called superset/prod with the following JSON structure:

{
  "DATABASE_URL": "postgresql://superset_user:password@rds-endpoint.region.rds.amazonaws.com:5432/superset",
  "SUPERSET_SECRET_KEY": "your-long-random-secret-key-min-32-chars",
  "REDIS_URL": "redis://redis-endpoint:6379/0",
  "SQLALCHEMY_TRACK_MODIFICATIONS": "False",
  "FLASK_ENV": "production"
}

The DATABASE_URL points to your RDS instance. The SUPERSET_SECRET_KEY is used by Flask for session signing; generate it with python -c "import secrets; print(secrets.token_hex(32))". The REDIS_URL is optional but recommended for caching and task queue support.

Also store data source credentials (e.g., for Snowflake, BigQuery, Postgres) as separate secrets or within the same secret, depending on your preference. For example:

{
  "SNOWFLAKE_ACCOUNT": "xy12345.us-east-1",
  "SNOWFLAKE_USER": "superset_service_account",
  "SNOWFLAKE_PASSWORD": "encrypted_password",
  "SNOWFLAKE_WAREHOUSE": "COMPUTE_WH"
}

IAM Role for Fargate Task Execution

Create an IAM role superset-task-execution-role with the following policies:

AmazonECSTaskExecutionRolePolicy (AWS managed): Allows ECS to pull images from ECR, write logs to CloudWatch, and retrieve secrets from Secrets Manager.

Custom inline policy for Secrets Manager access:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue"
      ],
      "Resource": "arn:aws:secretsmanager:region:account-id:secret:superset/prod-*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt"
      ],
      "Resource": "arn:aws:kms:region:account-id:key/key-id"
    }
  ]
}

This allows the task to fetch secrets at startup. If you’re using a KMS key to encrypt secrets (recommended), include the KMS decrypt permission.

Environment Variables and Configuration Files

Superset configuration can be provided via environment variables or a Python config file. For Fargate, environment variables are simpler:

SUPERSET_LOAD_EXAMPLES=false
SUPERSET_ROW_LIMIT=10000
SUPERSET_SQLLAB_TIMEOUT=600
SUPERSET_RESULTS_BACKEND_USE_MSGPACK=true

Pass these in the ECS task definition’s environment array. For more complex configuration (e.g., custom authentication, plugin configuration), mount a config file from EFS or embed it in the Docker image.

---

Database and Persistent Storage

RDS for Superset Metadata

Create a PostgreSQL RDS instance:

• Engine: PostgreSQL 13 or later
• Instance class: db.t3.small or db.t3.medium for most deployments (adjust based on dashboard complexity and user count)
• Storage: 20–50 GB, enable autoscaling to avoid running out of space
• Multi-AZ: Enabled for production (automatic failover if primary fails)
• Backup retention: 7 days minimum
• VPC and security group: Place in private subnets, use sg-rds
• Database name: superset
• Master username: superset_user (or similar)
• Password: Generate a strong password and store in Secrets Manager

Once the RDS instance is available, initialise the Superset metadata schema. You’ll do this the first time a Fargate task starts; Superset will run migrations automatically if the database is empty.

EFS for Persistent Configuration and Logs

Create an EFS file system in the same VPC:

• Performance mode: General purpose
• Throughput mode: Bursting
• Availability zones: Span both AZs where your Fargate tasks run

Create mount targets in both private subnets. Create a security group for EFS (sg-efs) allowing NFS traffic (port 2049) from the Fargate security group.

Within EFS, create directories:

/superset/config
/superset/logs
/superset/plugins

Mount EFS at /var/lib/superset in your Fargate task (see Task Definition section below). This persists Superset’s configuration, custom plugins, and logs across task restarts.

Alternatively, use S3 for logs and backups, with a Lambda function that periodically exports RDS snapshots to S3. This reduces EFS costs but adds operational complexity.

---

Task Definition and Container Configuration

The ECS task definition is the blueprint for your Fargate tasks. It specifies the Docker image, resource allocation, environment variables, logging, and volume mounts.

Task Definition JSON

Create a task definition called superset-prod (version 1 initially):

{
  "family": "superset-prod",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "1024",
  "memory": "2048",
  "executionRoleArn": "arn:aws:iam::account-id:role/superset-task-execution-role",
  "taskRoleArn": "arn:aws:iam::account-id:role/superset-task-role",
  "containerDefinitions": [
    {
      "name": "superset",
      "image": "apache/superset:latest",
      "portMappings": [
        {
          "containerPort": 8088,
          "protocol": "tcp"
        }
      ],
      "essential": true,
      "environment": [
        {
          "name": "SUPERSET_LOAD_EXAMPLES",
          "value": "false"
        },
        {
          "name": "SUPERSET_ROW_LIMIT",
          "value": "10000"
        },
        {
          "name": "SUPERSET_SQLLAB_TIMEOUT",
          "value": "600"
        },
        {
          "name": "FLASK_ENV",
          "value": "production"
        }
      ],
      "secrets": [
        {
          "name": "DATABASE_URL",
          "valueFrom": "arn:aws:secretsmanager:region:account-id:secret:superset/prod:DATABASE_URL::"
        },
        {
          "name": "SUPERSET_SECRET_KEY",
          "valueFrom": "arn:aws:secretsmanager:region:account-id:secret:superset/prod:SUPERSET_SECRET_KEY::"
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/superset-prod",
          "awslogs-region": "region",
          "awslogs-stream-prefix": "ecs"
        }
      },
      "mountPoints": [
        {
          "sourceVolume": "superset-efs",
          "containerPath": "/var/lib/superset",
          "readOnly": false
        }
      ]
    }
  ],
  "volumes": [
    {
      "name": "superset-efs",
      "efsVolumeConfiguration": {
        "fileSystemId": "fs-xxxxxxxxx",
        "transitEncryption": "ENABLED",
        "authorizationConfig": {
          "accessPointId": "fsap-xxxxxxxxx"
        }
      }
    }
  ]
}

Key points:

• networkMode: awsvpc is required for Fargate; the task gets its own ENI (Elastic Network Interface).
• requiresCompatibilities: ["FARGATE"] specifies this task runs on Fargate, not EC2.
• cpu and memory must be valid Fargate combinations (e.g., 1024 CPU with 2048–8192 MB memory). Start with 1024 CPU and 2048 MB; scale up if tasks are memory-constrained.
• secrets are injected from Secrets Manager at task startup. The format valueFrom references the secret ARN and JSON key.
• logConfiguration sends container logs to CloudWatch Logs. Create the log group /ecs/superset-prod beforehand.
• mountPoints attach EFS to the container. The fileSystemId is from your EFS resource; optionally use an access point for finer-grained permissions.

Container Startup and Initialisation

The official Superset image includes an entrypoint that runs migrations and starts the Flask app. On first startup, it will initialise the RDS database. Subsequent startups are faster.

If you need custom initialisation (e.g., loading a custom config file, registering data sources), create a custom Docker image:

FROM apache/superset:latest

COPY superset_config.py /app/superset_config.py
COPY entrypoint.sh /entrypoint.sh

RUN chmod +x /entrypoint.sh

ENTRYPOINT ["/entrypoint.sh"]

Your entrypoint.sh can run custom setup before calling the original entrypoint:

#!/bin/bash
set -e

# Run Superset migrations
superset db upgrade

# Create default admin user if it doesn't exist
superset fab create-admin --username admin --firstname Admin --lastname User --email admin@example.com --password $ADMIN_PASSWORD || true

# Start Superset
exec superset run -h 0.0.0.0 -p 8088 --with-threads --reload --debugger

Push this image to ECR (Elastic Container Registry) and reference it in the task definition.

---

Service Deployment and Autoscaling

Once your task definition is ready, create an ECS service to run and manage Fargate tasks.

ECS Service Configuration

Create a service called superset-prod:

• Cluster: Create a new cluster (or use an existing one) called analytics
• Task definition: superset-prod:1
• Desired count: Start with 2 tasks (for high availability across AZs)
• Deployment configuration: Rolling update, minimum healthy percent 50%, maximum percent 200% (allows one task to be replaced at a time)
• Network configuration:
  • VPC: Your VPC
  • Subnets: Both private subnets (one per AZ)
  • Security group: sg-fargate
  • Public IP: Disabled (tasks are behind ALB)
• Load balancing:
  • Target group: The ALB target group you created earlier
  • Container: superset (from task definition)
  • Port: 8088

Autoscaling Policies

Create an Application Auto Scaling target for the service:

• Service: analytics/superset-prod
• Min capacity: 2 tasks
• Max capacity: 10 tasks (adjust based on expected load and budget)

Attach scaling policies:

Scale-up policy (target tracking):

• Metric: ECSServiceAverageCPUUtilisation
• Target value: 70%
• Scale-out cooldown: 300 seconds

Scale-down policy (target tracking):

• Metric: ECSServiceAverageMemoryUtilisation
• Target value: 80%
• Scale-in cooldown: 600 seconds (longer to avoid thrashing)

Alternatively, use step scaling for finer control:

• If CPU > 80% for 2 minutes: add 2 tasks
• If CPU > 90% for 1 minute: add 4 tasks
• If CPU < 30% for 5 minutes: remove 1 task

Test autoscaling by generating load (e.g., Apache Bench or a load testing tool) and observing the service scale up and down.

---

Observability, Logging, and Monitoring

Production Superset requires visibility into application health, query performance, and resource utilisation.

CloudWatch Logs

All container logs go to /ecs/superset-prod log group. Create a log retention policy (e.g., 7 days) to control costs:

aws logs put-retention-policy \
  --log-group-name /ecs/superset-prod \
  --retention-in-days 7

Create CloudWatch Insights queries to diagnose issues:

Find slow queries:

fields @timestamp, @message
| filter @message like /query.*duration/
| stats avg(duration) as avg_duration by queryId
| sort avg_duration desc

Find errors:

fields @timestamp, @message
| filter @message like /ERROR|Exception|Traceback/
| stats count() as error_count by @message

CloudWatch Metrics and Alarms

Create alarms for:

ECS Service Health:

• RunningCount < DesiredCount for 5 minutes → SNS notification (tasks failing to start)
• CPUUtilisation > 90% for 5 minutes → page (capacity issue)
• MemoryUtilisation > 85% for 5 minutes → page (memory leak or undersizing)

ALB Health:

• UnHealthyHostCount > 0 for 2 minutes → SNS notification (task health check failing)
• TargetResponseTime > 2000ms for 5 minutes → SNS notification (slow queries or database issue)

RDS Health:

• DatabaseConnections > 80 → SNS notification (connection pool exhaustion)
• CPUUtilisation > 80% → SNS notification (database overload)

Create an SNS topic superset-alerts and subscribe your on-call team’s email or Slack webhook.

Custom Application Metrics

Superset exposes Prometheus-style metrics at /metrics if the Prometheus plugin is installed. Scrape this endpoint with CloudWatch Container Insights or a dedicated monitoring tool (Datadog, New Relic) to track:

• Active database connections
• Query execution time (p50, p95, p99)
• Cache hit rate
• Dashboard load time

For teams building analytics-heavy platforms, consider integrating Superset metrics with your platform engineering strategy to track user adoption and performance SLOs.

---

Operational Runbooks

Production systems need documented procedures for common tasks and incidents.

Deploying a New Superset Version

1. Test locally: Build and test the new Superset image on your laptop or in a dev environment.
2. Tag and push to ECR:

   docker tag superset:new-version 123456789.dkr.ecr.region.amazonaws.com/superset:new-version
   docker push 123456789.dkr.ecr.region.amazonaws.com/superset:new-version

3. Create a new task definition revision: Update the task definition to reference the new image tag. AWS will create a new revision (e.g., superset-prod:2).
4. Update the ECS service:

   aws ecs update-service \
     --cluster analytics \
     --service superset-prod \
     --task-definition superset-prod:2 \
     --force-new-deployment

5. Monitor the deployment: Watch the ECS service dashboard. The rolling update will replace tasks one at a time. Check CloudWatch Logs and ALB target health during the rollout.
6. Verify: Once all tasks are running the new version, test the application (log in, run a query, view a dashboard).
7. Rollback if needed:

   aws ecs update-service \
     --cluster analytics \
     --service superset-prod \
     --task-definition superset-prod:1 \
     --force-new-deployment

Scaling Manually

If autoscaling isn’t keeping up (e.g., during a spike):

aws ecs update-service \
  --cluster analytics \
  --service superset-prod \
  --desired-count 5

This immediately scales to 5 tasks. Autoscaling will take over once the metric-based policies kick in.

Database Maintenance

RDS automatically backs up your database. To restore from a backup:

1. Create a new RDS instance from the backup (restore point in time).
2. Update the DATABASE_URL secret to point to the new instance.
3. Restart the Fargate tasks (force a new deployment) to pick up the new secret.

For routine maintenance (e.g., vacuuming, index rebuilds), schedule a Lambda function to run during off-peak hours.

Handling a Task Crash Loop

If tasks are crashing and restarting repeatedly:

1. Check CloudWatch Logs for errors. Common causes:
   • Secrets Manager credentials wrong or inaccessible
   • Database connection string invalid
   • Out of memory (increase task memory in task definition)
   • Missing environment variables
2. If the issue is in the application code, roll back to the previous task definition revision.
3. Fix the issue, rebuild the image, and redeploy.

To prevent crash loops from affecting users, set the service’s minimum healthy percent to 50% so at least one task remains running while others restart.

Clearing the Cache

If Superset’s cache is stale or corrupted:

1. If using Redis, flush the cache:

   redis-cli FLUSHDB

2. If using the file-based cache (on EFS), remove the cache directory:

   rm -rf /var/lib/superset/cache/*

3. Restart the Fargate tasks to pick up the cleared cache.

---

Cost Optimisation

Fargate is billed per vCPU-hour and GB-hour. To keep costs reasonable:

Right-size Tasks

Start with 1024 CPU (0.25 vCPU) and 2048 MB memory. Monitor actual usage:

aws cloudwatch get-metric-statistics \
  --namespace AWS/ECS \
  --metric-name CPUUtilisation \
  --dimensions Name=ServiceName,Value=superset-prod Name=ClusterName,Value=analytics \
  --start-time 2024-01-01T00:00:00Z \
  --end-time 2024-01-31T23:59:59Z \
  --period 3600 \
  --statistics Average,Maximum

If average CPU is < 30% and memory < 1024 MB, scale down. If hitting limits, scale up.

Use Fargate Spot

For non-critical workloads or dev/test environments, use Fargate Spot (up to 70% discount). Add a Spot capacity provider to your ECS cluster and configure the service to use it. Be aware that Spot tasks can be interrupted with 2 minutes’ notice, so keep your desired count high enough that losing a task doesn’t impact availability.

Optimise Data Transfer

• Use VPC endpoints for S3 and Secrets Manager to avoid NAT gateway charges (which are ~$0.045/GB).
• Cache frequently accessed data locally (in EFS or Redis) to reduce queries to external data sources.
• Use AWS PrivateLink if querying data sources in other AWS accounts.

Reserved Capacity

If your Superset deployment runs 24/7, consider Savings Plans (compute) or Reserved Instances (RDS). For example, a 1-year Savings Plan for 1 vCPU is ~30% cheaper than on-demand.

Monitor Costs

Set up a CloudWatch alarm on your ECS service’s estimated cost:

aws ce get-cost-and-usage \
  --time-period Start=2024-01-01,End=2024-01-31 \
  --granularity DAILY \
  --metrics BlendedCost \
  --group-by Type=DIMENSION,Key=SERVICE

If costs spike, investigate:

• Autoscaling adding too many tasks (check CPU/memory metrics)
• Data transfer costs (check NAT gateway or data source queries)
• RDS instance oversized (check database CPU/connections)

---

Next Steps

You now have a production-ready Superset deployment on ECS Fargate. The architecture scales with your data and users, logs are centralised in CloudWatch, and alarms alert your team to issues.

From here, consider:

Data Source Integration: Connect Superset to your data warehouse (Snowflake, BigQuery, Postgres, ClickHouse). This is where the value comes in—dashboards and queries run against your actual data. For teams modernising analytics infrastructure, this is often paired with a data platform re-architecture. If you’re building a multi-tenant SaaS platform with embedded analytics, Superset integrates seamlessly with ClickHouse or Postgres backends.

Custom Plugins and Themes: Superset is extensible. Build custom visualisations, authentication backends, or themes to match your brand. The official Superset documentation covers plugin development.

High Availability and Disaster Recovery: This guide covers basic HA (multi-AZ, autoscaling). For mission-critical deployments, add:

• Multi-region RDS read replicas for failover
• Cross-region ECS service replication
• Automated backup and restore testing

Security Hardening: Superset supports LDAP, SAML, and OAuth authentication. Integrate with your identity provider. Enable row-level security (RLS) if different users should see different data. Audit query execution and dashboard access via CloudTrail and application logs.

Performance Tuning: As you scale, monitor query performance. Superset caches query results; configure cache TTLs based on your data freshness requirements. Use Superset’s query profiler to identify slow queries and optimise them at the data source level.

Observability at Scale: For teams running analytics across multiple regions or serving hundreds of dashboards, integrate Superset with your observability stack. Tools like Datadog or New Relic provide deeper insights into application performance and user behaviour.

If you’re building a data-driven platform or modernising your analytics infrastructure, this Superset on Fargate pattern is a solid foundation. Whether you’re in Australia or the United States, the operational principles remain the same: keep it simple, monitor everything, automate routine tasks, and iterate based on real usage.

For teams needing hands-on support shipping analytics platforms, platform engineering services can accelerate time-to-value. If you’re unsure whether Superset is the right fit for your data stack, a quick AI Quickstart Audit can help clarify your options and priorities.

---

Summary

This guide walked through a complete production deployment of Apache Superset on AWS ECS Fargate:

1. Architecture: Stateless Fargate tasks, PostgreSQL RDS for metadata, EFS for persistent config, ALB for traffic distribution, autoscaling for demand.
2. Networking: VPC with public/private subnets, security groups enforcing least-privilege, NAT for outbound internet access.
3. Secrets: Secrets Manager for database credentials and API keys, injected at task startup.
4. Compute: Task definitions with resource limits, container logs to CloudWatch, health checks for load balancer.
5. Operations: Deployment procedures, scaling runbooks, monitoring and alerting, cost optimisation.

The pattern is battle-tested across dozens of deployments. The operational habits—monitoring, alerting, automated rollback, runbooks—are what separate a working deployment from one that stays working when things go wrong.

Start with the minimum viable setup (2 tasks, t3.small RDS, 20 GB EFS), monitor real usage, and scale from there. Superset is lightweight; most deployments never exceed 4–6 Fargate tasks or a medium RDS instance. The biggest cost is usually data transfer, not compute.

For additional context on building analytics platforms or modernising your data stack, explore platform engineering services in your region or across the US. If you need help with the full deployment—from architecture review to production handoff—that’s what platform engineering teams do.