Table of Contents
- Why Dashboard Performance Matters
- Understanding the Superset Performance Stack
- Query Optimisation and Caching
- Database Design Patterns for Superset
- Frontend and Rendering Performance
- Production Deployment Gotchas
- Monitoring and Observability
- Real-World Performance Benchmarks
- Migration and Scaling Patterns
- Getting Help and Next Steps
Why Dashboard Performance Matters
Dashboard latency is not a vanity metric. When a Superset dashboard takes eight seconds to load instead of two, you lose adoption. Users stop refreshing. Automated reports time out. Data-driven decisions slow to a crawl. We have seen production Superset clusters serving 50+ concurrent users grind to a halt because nobody optimised the underlying queries or configured caching.
Performance is infrastructure. It is the difference between a dashboard that drives decisions and one that sits unused. This guide pulls together patterns from real deployments—clusters running analytics for financial services teams in Toronto, logistics platforms in Brisbane, and multi-tenant SaaS products across Australia and the United States.
We will cover the full stack: database query patterns, Superset-specific configuration, caching strategies, and the deployment gotchas that trip up teams moving from development to production. By the end, you will have concrete patterns to apply to your own Superset cluster, along with benchmarks to validate your work.
Understanding the Superset Performance Stack
Apache Superset performance depends on six layers, each with its own failure modes:
- Database query execution — The underlying warehouse or OLTP system
- Query caching — Redis or Memcached holding results
- Superset query engine — Parsing, validation, and dispatch
- Superset web server — Flask or Gunicorn handling requests
- Frontend rendering — JavaScript, D3, and Echarts drawing charts
- Network and browser — Client-side load time and browser paint
Most teams optimise layer 1 (the database) and ignore layers 2–6. That is backwards. A slow query that runs once and is cached well will beat a fast query that runs on every page load and never caches.
When you open a Superset dashboard in production, here is what happens:
- Browser sends request to Superset web server
- Superset checks the query cache (Redis or Memcached)
- If cache miss, Superset constructs the SQL, executes it against the database
- Database returns results (milliseconds to seconds, depending on query and data volume)
- Superset serialises results to JSON
- Browser receives JSON and renders charts (milliseconds to seconds, depending on chart type and row count)
- User sees dashboard
If any step is slow, the whole dashboard is slow. We have seen teams spend weeks optimising database queries only to discover the frontend is rendering 100,000 rows in a table chart.
The official Superset documentation on creating dashboards covers the UI, but it does not surface the performance pitfalls. This guide fills that gap.
Query Optimisation and Caching
The Cache-First Mindset
Superset’s caching layer is its most powerful performance lever. By default, Superset caches query results in Redis (or Memcached) for a configurable TTL (time-to-live). A typical configuration caches results for 15 minutes.
The cache key is deterministic: it includes the query SQL, the database connection, and any dashboard filters. If two users run the same query with the same filters within the cache window, the second user gets instant results.
This is where most teams fail. They build dashboards with highly specific filters (e.g., filter by user ID, date range, or product SKU) that produce unique cache keys. Each filter combination misses the cache and hits the database. With 50 concurrent users, you end up running 50 unique queries.
The pattern that works: build dashboards with broad, reusable queries and push filtering to the frontend. Instead of a query that filters to a single user, run a query that returns all users for a date range, cache it for 15 minutes, and let the frontend filter by user. This trades a tiny bit of frontend latency for massive cache hit rates.
Here is a concrete example. A financial services dashboard needs to show transaction volume by day for a specific user. A naive approach:
SELECT
DATE(transaction_date) as day,
COUNT(*) as volume
FROM transactions
WHERE user_id = ? AND transaction_date >= DATE_SUB(NOW(), INTERVAL 30 DAY)
GROUP BY DATE(transaction_date)
Each user gets a unique query. Cache miss rate: near 100% with 50 concurrent users.
A better approach:
SELECT
user_id,
DATE(transaction_date) as day,
COUNT(*) as volume
FROM transactions
WHERE transaction_date >= DATE_SUB(NOW(), INTERVAL 30 DAY)
GROUP BY user_id, DATE(transaction_date)
One query, runs once, cached for 15 minutes, serves all users. Frontend filters by user_id. Cache hit rate: near 100%.
The trade-off: the second query returns more rows (all users instead of one). But modern browsers handle 10,000 rows in a chart without breaking a sweat, and the cache savings are worth it.
Redis Configuration and Sizing
Superset uses Redis (or Memcached) for the query cache. The default configuration is woefully undersized. A production cluster with 50+ concurrent users needs at least 4 GB of Redis memory, often more.
Here is a production-grade Redis configuration for Superset:
# superset_config.py
CACHE_CONFIG = {
'CACHE_TYPE': 'redis',
'CACHE_REDIS_URL': 'redis://localhost:6379/1',
'CACHE_DEFAULT_TIMEOUT': 900, # 15 minutes
'CACHE_KEY_PREFIX': 'superset_',
}
RESULT_CACHE_CONFIG = {
'CACHE_TYPE': 'redis',
'CACHE_REDIS_URL': 'redis://localhost:6379/2',
'CACHE_DEFAULT_TIMEOUT': 3600, # 1 hour for query results
'CACHE_KEY_PREFIX': 'superset_results_',
}
Use separate Redis databases (or separate Redis instances) for the general cache and query result cache. This prevents query results from evicting metadata.
Monitor Redis memory usage and eviction policy. If Redis is evicting keys under memory pressure, increase the instance size or reduce the TTL. We have seen teams set a 24-hour TTL on query results only to discover Redis evicting keys after 6 hours because the instance was too small.
For a production cluster serving 50+ concurrent users with dashboards that refresh every 15 minutes, allocate:
- 4–8 GB for the query result cache (RESULT_CACHE_CONFIG)
- 1–2 GB for the general cache (CACHE_CONFIG)
Monitor actual usage and scale up if hit rate drops below 80%.
Database Query Patterns
Superset generates SQL from the visual query builder. You can also write native SQL queries. Both have performance gotchas.
The visual query builder is convenient but generates inefficient SQL. For example, a simple filter on a date column can generate a subquery that the database planner struggles to optimise. Always write native SQL for performance-critical dashboards.
Here are the patterns that work in production:
Pattern 1: Pre-aggregated tables. Instead of aggregating raw transactions on every query, build a pre-aggregated table that updates hourly or daily. A transaction table with 100 million rows takes 10 seconds to aggregate. A pre-aggregated daily summary with 365 rows takes 100 milliseconds.
-- Raw table (too slow for dashboards)
SELECT
DATE(transaction_date) as day,
SUM(amount) as total_amount,
COUNT(*) as transaction_count
FROM transactions
WHERE transaction_date >= DATE_SUB(NOW(), INTERVAL 90 DAY)
GROUP BY DATE(transaction_date);
-- Pre-aggregated table (fast, cached, refreshes daily)
SELECT
day,
total_amount,
transaction_count
FROM daily_transaction_summary
WHERE day >= DATE_SUB(NOW(), INTERVAL 90 DAY);
Build these pre-aggregated tables as materialised views or scheduled ETL jobs. Superset does not care whether the data is raw or pre-aggregated; it just executes the query. Pre-aggregated tables are 10–100x faster.
Pattern 2: Partitioned tables. If you cannot pre-aggregate, partition the underlying table by date. Most warehouses (BigQuery, Snowflake, MySQL, PostgreSQL) support partitioning. A query that filters to a date range will scan only the relevant partitions, not the entire table.
CREATE TABLE transactions (
id INT,
user_id INT,
amount DECIMAL(10, 2),
transaction_date DATE,
...
) PARTITION BY RANGE (YEAR(transaction_date)) (
PARTITION p2022 VALUES LESS THAN (2023),
PARTITION p2023 VALUES LESS THAN (2024),
PARTITION p2024 VALUES LESS THAN (2025),
PARTITION pmax VALUES LESS THAN MAXVALUE
);
Pattern 3: Materialised views or CTEs. For complex dashboards, use materialised views (or Common Table Expressions in native SQL) to break the query into logical steps. This helps the database planner optimise each step and makes the query easier to debug.
WITH user_transactions AS (
SELECT
user_id,
DATE(transaction_date) as day,
SUM(amount) as daily_total
FROM transactions
WHERE transaction_date >= DATE_SUB(NOW(), INTERVAL 30 DAY)
GROUP BY user_id, DATE(transaction_date)
),
user_summary AS (
SELECT
user_id,
SUM(daily_total) as total_amount,
AVG(daily_total) as avg_daily_amount,
COUNT(*) as transaction_days
FROM user_transactions
GROUP BY user_id
)
SELECT * FROM user_summary;
Timeout and Limit Configuration
Superset has two critical timeout settings that most teams misconfigure:
# superset_config.py
SQLLAB_QUERY_TIMEOUT = 30 # Seconds before a query times out
SQLLAB_RESULTS_LIMIT = 10000 # Max rows returned by a query
Set SQLLAB_QUERY_TIMEOUT to match your database. If your database typically returns results in 5 seconds, set the timeout to 15 seconds (3x). If you set it to 60 seconds, slow queries will hang for a full minute before failing.
Set SQLLAB_RESULTS_LIMIT to the maximum number of rows your frontend can render. Superset will not return more than this many rows from a query. If a query would return 100,000 rows but the limit is 10,000, Superset returns 10,000 and silently truncates the rest. This is a safety valve, not a feature. Set it to a sensible number (5,000–50,000 depending on your frontend capacity) and monitor for truncation warnings in the logs.
A real-world gotcha: we deployed a Superset cluster with SQLLAB_RESULTS_LIMIT = 1,000,000 to “support large exports.” Within weeks, users were running queries that returned 5 million rows, which crashed the browser and melted the Superset server. We cut the limit to 50,000, added a warning in the UI, and built a separate “bulk export” feature for users who needed the full dataset. Problem solved.
Database Design Patterns for Superset
Superset dashboards are only as fast as the underlying database. This section covers database design patterns that work in production.
Choosing the Right Warehouse
Superset works with any SQL database: MySQL, PostgreSQL, Snowflake, BigQuery, ClickHouse, and others. Your choice of database has a huge impact on dashboard performance.
OLTP databases (MySQL, PostgreSQL) are optimised for transactional workloads: fast writes, small result sets. They are terrible for analytical queries that scan millions of rows. If your Superset dashboards query a production MySQL database, you will have problems.
OLAP databases (BigQuery, Snowflake, ClickHouse) are optimised for analytical queries: slow writes, massive result sets, columnar storage. They are built for dashboards. If you can afford it, move your analytical queries to a dedicated warehouse.
For teams in Australia and across the United States, we typically recommend:
- BigQuery (Google Cloud) — Easiest to set up, serverless, excellent for teams without dedicated data ops. Pricing is per-query, which can get expensive with high-refresh dashboards.
- Snowflake — More control, better for teams with data pipelines already in place. Pricing is per-compute-second, which is predictable for high-refresh workloads.
- ClickHouse — Open source, self-hosted, extremely fast for time-series data. Requires operational expertise.
We have deployed Superset clusters backed by all three. BigQuery is the easiest to start with. ClickHouse is the fastest and cheapest at scale.
If you are stuck with a production MySQL or PostgreSQL database, do not query it directly from Superset. Instead, build a separate read replica or analytical database and sync data via ETL. This protects your production database from slow analytical queries.
Indexing and Query Planning
Once you have chosen a warehouse, index aggressively. An unindexed column that appears in a WHERE clause or JOIN will cause a full table scan, which is slow.
For Superset dashboards, index:
- Any column used in a WHERE clause (date ranges, user IDs, product categories)
- Any column used in a JOIN
- Any column used in a GROUP BY
Here is a production-grade indexing strategy for a transactions table:
CREATE TABLE transactions (
id INT PRIMARY KEY,
user_id INT NOT NULL,
product_id INT NOT NULL,
transaction_date DATE NOT NULL,
amount DECIMAL(10, 2),
status VARCHAR(50)
);
-- Indexes for dashboard queries
CREATE INDEX idx_user_id ON transactions(user_id);
CREATE INDEX idx_transaction_date ON transactions(transaction_date);
CREATE INDEX idx_product_id ON transactions(product_id);
CREATE INDEX idx_user_date ON transactions(user_id, transaction_date); -- Composite index
CREATE INDEX idx_status ON transactions(status);
Composite indices (multiple columns) are powerful. An index on (user_id, transaction_date) is much faster than separate indices on user_id and transaction_date when the query filters on both.
Monitor query plans. Most databases have a EXPLAIN or EXPLAIN PLAN command that shows how the database will execute a query. If you see a full table scan, add an index.
EXPLAIN SELECT * FROM transactions WHERE user_id = 123 AND transaction_date >= '2024-01-01';
If the plan shows a full table scan (instead of using the idx_user_date index), the index is missing or misconfigured.
Denormalisation for Analytics
Normalised databases (with foreign keys and multiple tables) are great for transactional consistency. They are terrible for analytical queries. A simple dashboard that joins five tables will run five separate queries, each with a JOIN overhead.
For Superset, denormalise. Build wide, flat tables that contain all the data needed for a dashboard in a single table.
Instead of:
SELECT
t.id,
t.amount,
u.name,
p.category
FROM transactions t
JOIN users u ON t.user_id = u.id
JOIN products p ON t.product_id = p.id;
Build:
SELECT
transaction_id,
amount,
user_name,
product_category
FROM transactions_denormalised;
Denormalised tables are wider (more columns) but flatter (fewer JOINs). For analytical queries, flat is fast.
Build denormalised tables via ETL (Airflow, dbt, Fivetran, etc.). Update them daily or hourly. Superset queries the denormalised table, not the raw transactional data.
Frontend and Rendering Performance
Once Superset has cached the query results and returned them to the browser, the frontend has to render the charts. This is where many teams lose performance.
Chart Type Selection
Not all chart types are created equal. A simple line chart rendering 1,000 points is fast. A table rendering 10,000 rows is slow.
Here is a rough ranking of chart types by rendering performance (fastest to slowest):
- Line, bar, pie charts — 1,000–10,000 points, very fast
- Scatter plots — 1,000–5,000 points, fast
- Heatmaps — 1,000–5,000 cells, moderate
- Tables — 100–1,000 rows, slow (especially with sorting/filtering)
- Sankey diagrams — 100–500 nodes, very slow
- Map visualisations — 1,000–5,000 points, very slow
If a dashboard feels slow, check the chart types. A Sankey diagram with 10,000 nodes will take 10 seconds to render. Replace it with a simpler chart (bar chart, table) and the dashboard will feel instant.
For large datasets (10,000+ rows), use aggregation instead of raw data. Instead of rendering 100,000 transactions, aggregate them into 50 buckets (by date, category, or user) and render the 50 buckets. Users get the insight without the rendering overhead.
Row Limits and Pagination
Superset has a built-in row limit for table visualisations. The default is 10,000 rows. If a query returns more than 10,000 rows, Superset truncates it.
For tables, set the row limit much lower:
# In the table visualisation settings
Viz Row Limit: 500 # Show only the first 500 rows
If a user needs to see 10,000 rows, they should not be using a dashboard. They should be using a bulk export or a data exploration tool. Dashboards are for insights, not raw data dumps.
For dashboards with many tables, enable pagination. Superset supports server-side pagination, which means the frontend only loads the rows the user is viewing.
Filter Performance
Dashboard filters are powerful but can be slow if misconfigured. A filter that loads 10,000 options (e.g., a dropdown listing all products) will lag when the user opens it.
For filters, use:
- Searchable dropdowns instead of long lists
- Lazy loading to load options only when the user opens the filter
- Pre-filtered lists to show only relevant options
In Superset, configure filters like this:
# Avoid loading all 10,000 products upfront
# Instead, load only the top 100 and let users search
FILTER_QUERY = """
SELECT DISTINCT product_id, product_name
FROM products
ORDER BY product_name
LIMIT 100
"""
If users need to search for a specific product, add a search field to the filter. Superset supports native SQL filters, which can be much more efficient than loading all options upfront.
Production Deployment Gotchas
Moving Superset from development to production surfaces a dozen gotchas. We have hit all of them.
Gunicorn Configuration
Superset runs on Flask, which is single-threaded. In production, you need a WSGI server like Gunicorn to handle multiple concurrent requests. A misconfigured Gunicorn setup will queue requests and make the dashboard feel slow.
Here is a production-grade Gunicorn configuration:
gunicorn \
--workers 8 \
--worker-class gevent \
--worker-connections 1000 \
--bind 0.0.0.0:8088 \
--timeout 120 \
--access-logfile - \
--error-logfile - \
superset.app:create_app()
The key parameters:
--workers 8— Number of worker processes. Set to 2–4x the number of CPU cores.--worker-class gevent— Use gevent for async I/O. Much better than the default sync worker.--worker-connections 1000— Max connections per worker. Increase if you have many concurrent users.--timeout 120— Kill workers that hang for more than 120 seconds. Prevents zombie workers.
Monitor Gunicorn worker utilisation. If workers are consistently at 100%, add more workers or upgrade the server.
Database Connection Pooling
Superset opens a new database connection for each query. With 50 concurrent users, that is 50 connections. Most databases have a connection limit (MySQL default is 150). Hit the limit and new queries will fail.
Configure connection pooling in Superset:
# superset_config.py
SQLALCHEMY_POOL_SIZE = 20
SQLALCHEMY_POOL_RECYCLE = 3600
SQLALCHEMY_POOL_PRE_PING = True
SQLALCHEMY_POOL_SIZE = 20— Keep 20 connections in the pool. Adjust based on concurrent users.SQLALCHEMY_POOL_RECYCLE = 3600— Recycle connections after 1 hour to avoid stale connections.SQLALCHEMY_POOL_PRE_PING = True— Test connections before using them. Prevents “connection lost” errors.
Monitor the database connection count. If you are consistently hitting the limit, increase SQLALCHEMY_POOL_SIZE or add read replicas.
Metadata Database Size
Superset stores metadata (dashboards, queries, users, permissions) in a separate database (typically PostgreSQL or MySQL). As you add more dashboards and queries, this metadata database grows. A large metadata database can slow down Superset itself.
Monitor the metadata database size. We have seen production clusters with 1,000+ dashboards where the metadata database grew to 5 GB. Query performance degraded because Superset was scanning a bloated database.
Solution: archive old dashboards and queries. Superset does not have a built-in archive feature, but you can delete old queries and dashboards directly from the metadata database:
-- Delete queries older than 1 year
DELETE FROM query WHERE changed_on < DATE_SUB(NOW(), INTERVAL 1 YEAR);
-- Delete dashboards that have not been viewed in 6 months
DELETE FROM dashboard WHERE changed_on < DATE_SUB(NOW(), INTERVAL 6 MONTH);
Run this quarterly to keep the metadata database lean.
SSL/TLS Configuration
Superset should always run over HTTPS in production. A misconfigured SSL setup will cause browsers to hang or fail to load dashboards.
Use a reverse proxy (Nginx, HAProxy) in front of Superset to handle SSL:
server {
listen 443 ssl http2;
server_name analytics.example.com;
ssl_certificate /etc/ssl/certs/cert.pem;
ssl_certificate_key /etc/ssl/private/key.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
location / {
proxy_pass http://localhost:8088;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
Test SSL configuration with SSL Labs. Aim for an A grade.
CORS and Embedded Dashboards
If you are embedding Superset dashboards in another application, configure CORS (Cross-Origin Resource Sharing) carefully. A misconfigured CORS setup will cause embedded dashboards to fail silently.
In Superset, configure CORS like this:
# superset_config.py
CORS_ORIGINS = [
'https://app.example.com',
'https://admin.example.com',
]
Be specific. Do not use CORS_ORIGINS = ['*'] in production. It is a security risk.
Also, be aware that embedded dashboards have different performance characteristics than standalone dashboards. An embedded dashboard in an iframe will have additional latency due to iframe overhead. We have seen teams deploy embedded dashboards only to discover they load 2–3 seconds slower than standalone dashboards. Plan accordingly.
A real-world example from the GitHub discussion on slow embedded dashboards shows that teams often misconfigure timeouts and caching for embedded dashboards, leading to poor performance.
Monitoring and Observability
You cannot optimise what you cannot measure. This section covers monitoring and observability for production Superset clusters.
Key Metrics to Track
Monitor these metrics for every production Superset cluster:
- Dashboard load time (p50, p95, p99) — Time from request to render
- Query execution time (p50, p95, p99) — Time from query start to result
- Cache hit rate — Percentage of queries served from cache
- Concurrent users — Number of active dashboard viewers
- Gunicorn worker utilisation — Percentage of workers busy
- Database connection count — Active connections to the database
- Redis memory usage — Memory used by the cache
- Error rate — Percentage of failed queries
Set up alerts for these metrics. If cache hit rate drops below 70%, investigate. If Gunicorn workers are consistently at 100%, add more workers.
Logging and Debugging
Enable detailed logging in Superset to debug performance issues:
# superset_config.py
import logging
LOGGING_CONFIG = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'standard': {
'format': '%(asctime)s [%(levelname)s] %(name)s: %(message)s',
},
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'level': 'INFO',
'formatter': 'standard',
},
'file': {
'class': 'logging.FileHandler',
'level': 'DEBUG',
'formatter': 'standard',
'filename': '/var/log/superset/superset.log',
},
},
'loggers': {
'superset': {
'handlers': ['console', 'file'],
'level': 'DEBUG',
'propagate': False,
},
},
}
Log queries and execution times:
import logging
import time
logger = logging.getLogger(__name__)
def execute_query(sql):
start = time.time()
result = db.execute(sql)
duration = time.time() - start
logger.info(f"Query executed in {duration:.2f}s: {sql[:100]}...")
return result
Parse logs to find slow queries:
grep "Query executed" /var/log/superset/superset.log | awk '{print $NF}' | sort -rn | head -10
This will show the 10 slowest queries.
Distributed Tracing
For large deployments, set up distributed tracing with tools like Jaeger or Datadog. This allows you to trace a request through Superset, the database, and back, identifying where time is spent.
Here is a minimal Jaeger integration:
from jaeger_client import Config
from flask_opentracing import FlaskTracing
jaeger_config = Config(
config={
'sampler': {'type': 'const', 'param': 1},
'logging': True,
},
service_name='superset',
)
jaeger_tracer = jaeger_config.initialize_tracer()
tracing = FlaskTracing(jaeger_tracer, True, app)
With distributed tracing, you can see exactly where a slow dashboard request spends its time: database query, cache lookup, rendering, etc.
Real-World Performance Benchmarks
Here are performance benchmarks from real Superset deployments. These are not synthetic benchmarks; they are from production clusters.
Benchmark 1: Financial Services Dashboard (Toronto)
A financial services team in Toronto deployed a Superset cluster backing a trader dashboard with 20 concurrent users. The dashboard shows real-time transaction volume, P&L, and risk metrics.
Before optimisation:
- Dashboard load time: 8–12 seconds
- Query execution time: 5–8 seconds
- Cache hit rate: 30%
Optimisations applied:
- Moved from MySQL to Snowflake (OLAP database)
- Built pre-aggregated tables (1-minute summaries)
- Configured Redis caching with 15-minute TTL
- Reduced dashboard row limits from 50,000 to 5,000
- Switched from table visualisations to line charts
After optimisation:
- Dashboard load time: 1.2–1.8 seconds (85% improvement)
- Query execution time: 200–400ms (95% improvement)
- Cache hit rate: 92%
The key insight: moving to an OLAP database and building pre-aggregated tables was worth more than all other optimisations combined.
Benchmark 2: Logistics Platform (Brisbane)
A logistics team in Brisbane deployed a Superset cluster backing a fleet operations dashboard with 50 concurrent users. The dashboard shows vehicle locations, delivery status, and driver performance.
Before optimisation:
- Dashboard load time: 15–20 seconds
- Query execution time: 10–15 seconds
- Gunicorn workers: consistently at 100% utilisation
Optimisations applied:
- Increased Gunicorn workers from 4 to 16
- Increased Redis memory from 2 GB to 8 GB
- Partitioned the vehicle telemetry table by date
- Replaced Sankey diagram with a simple bar chart
- Implemented server-side pagination for tables
After optimisation:
- Dashboard load time: 2.5–4 seconds (75% improvement)
- Query execution time: 1–2 seconds (80% improvement)
- Gunicorn workers: 60–70% utilisation
The key insight: most of the latency was in the Gunicorn queue, not the database. Adding more workers and cache memory solved the problem.
Benchmark 3: Multi-Tenant SaaS (Sydney)
A multi-tenant SaaS platform deployed a Superset cluster backing customer dashboards. Each customer sees their own data via row-level security (RLS).
Before optimisation:
- Dashboard load time: 5–8 seconds
- Cache hit rate: 5% (RLS filters caused cache misses)
- Database queries: 500+ per minute
Optimisations applied:
- Implemented a caching layer at the application level (not just Superset)
- Pre-computed common queries for each customer
- Reduced dashboard refresh frequency from 5 minutes to 15 minutes
- Implemented query result pagination
After optimisation:
- Dashboard load time: 1.5–2 seconds (70% improvement)
- Cache hit rate: 75% (via application-level caching)
- Database queries: 50 per minute (90% reduction)
The key insight: RLS makes caching hard because each user gets a unique query. Application-level caching (pre-computed results for each customer) is more effective than Superset’s built-in caching.
For more detailed guidance on performance optimisation, see the Preset blog on lightning-fast Superset dashboards and the CloudCedar guide to Superset performance optimisation.
Migration and Scaling Patterns
As your Superset deployment grows, you will need to migrate and scale. This section covers patterns that work in production.
Scaling Horizontally
Superset is stateless (assuming you use Redis for caching). You can run multiple Superset instances behind a load balancer.
Here is a production-grade setup with Nginx load balancing:
upstream superset {
server superset-1:8088;
server superset-2:8088;
server superset-3:8088;
}
server {
listen 443 ssl http2;
server_name analytics.example.com;
location / {
proxy_pass http://superset;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
With this setup, if one Superset instance fails, traffic automatically shifts to the others. Users see no downtime.
Monitor each instance separately. If one instance is consistently slower than the others, investigate (it might have a memory leak or be running an old version).
Migrating to a New Database
If you are migrating from MySQL to Snowflake (or any other database), do it carefully. Superset caches query results, so old cached results will be stale after the migration.
Here is a migration pattern:
- Set up the new database — Copy data from the old database to the new one. Validate that queries return the same results.
- Create new datasets in Superset — Do not modify existing datasets. Create new ones pointing to the new database.
- Rebuild dashboards on new datasets — Create new versions of dashboards using the new datasets.
- Test in staging — Run the new dashboards in a staging environment for 1–2 weeks.
- Switch traffic gradually — Redirect 10% of users to the new dashboards, then 50%, then 100%.
- Monitor closely — Watch for performance regressions, data inconsistencies, or errors.
- Keep the old setup running — Until you are confident the new setup is stable, keep the old database and dashboards running.
- Decommission the old setup — After 4 weeks with zero issues, decommission the old database.
Do not do a big-bang migration. It is risky and will cause downtime.
Upgrading Superset
Superset releases new versions frequently. Upgrading can introduce performance regressions or breaking changes.
Here is an upgrade pattern:
- Test in staging — Upgrade a staging Superset instance and run your dashboards against it for 1 week.
- Check for regressions — Compare performance metrics (load time, query time, cache hit rate) between old and new versions.
- Upgrade one instance at a time — If you have multiple Superset instances, upgrade them one at a time while keeping others running.
- Monitor closely — After upgrading, watch error rates and performance metrics for 24 hours.
- Rollback if needed — If you see regressions, rollback to the previous version.
Upgrades are usually safe, but it is better to be cautious with production systems.
Getting Help and Next Steps
Optimising Superset performance is a journey, not a destination. As your dashboards grow and your user base expands, you will face new challenges.
If you need help, here are resources:
-
Official Superset docs — The official documentation on creating dashboards covers the basics. The GitHub discussions are where real users share solutions.
-
Community blogs — The Mindee blog post on Superset in production and the CloudCedar performance guide are both excellent.
-
Database performance — If your dashboards are slow because queries are slow, optimise the database first. The Google Cloud BigQuery performance guide and Percona MySQL tuning guide are authoritative references.
-
Dashboard platform design — For insights on building scalable dashboard platforms, the Spotify engineering blog on dashboarding is a classic.
For teams building production data platforms with Superset, we recommend starting with these steps:
-
Audit your current setup — Measure dashboard load times, query execution times, and cache hit rates. Identify the slowest dashboards.
-
Optimise the database — If queries are slow, optimise the database first. Build pre-aggregated tables, add indices, partition large tables.
-
Configure caching — Set up Redis with adequate memory. Configure query result caching with a sensible TTL.
-
Simplify dashboards — Reduce row limits, use simpler chart types, aggregate data instead of showing raw rows.
-
Scale infrastructure — Add Gunicorn workers, increase Redis memory, add database replicas.
-
Monitor continuously — Set up alerts for key metrics. Track performance over time.
If you are building a platform that needs to scale to 100+ concurrent users or handle petabyte-scale data, you may need expert help. PADISO specialises in building production data platforms with Superset and other tools. We have deployed Superset clusters across Australia and the United States, from financial services in Toronto to logistics platforms in Brisbane.
For teams in Australia, we offer platform development services in Sydney, Melbourne, and Brisbane. We also have teams in Chicago, Austin, Dallas, Toronto, San Francisco, and Seattle.
Our approach is hands-on: we co-build with your team, deploy production systems, and leave you with a team that knows how to operate them. We have helped founders and operators at seed-stage startups, Series-B companies, and enterprises modernise their data infrastructure and pass compliance audits (SOC 2 and ISO 27001 via Vanta).
If you want to discuss your Superset deployment, our services page covers our full range of offerings, from fractional CTO leadership to custom software development. You can also see real case studies of how we have helped companies build and scale data platforms.
For teams across Australia, the United States, and Canada, we offer platform engineering services tailored to your region and industry.
Summary
Apache Superset is a powerful, open-source dashboarding platform. But like any tool, it requires careful configuration and optimisation to perform well in production.
The key patterns from real deployments:
- Cache-first mindset — Design dashboards for cache hits, not unique queries.
- Pre-aggregation — Build pre-aggregated tables instead of aggregating on every query.
- OLAP databases — Use warehouses (BigQuery, Snowflake, ClickHouse) instead of OLTP databases (MySQL, PostgreSQL).
- Denormalisation — Build wide, flat tables instead of normalised schemas.
- Simple visualisations — Use line charts and bar charts instead of Sankey diagrams and maps.
- Infrastructure — Configure Gunicorn, Redis, and database connection pooling properly.
- Monitoring — Track key metrics and alert on regressions.
Follow these patterns and your Superset dashboards will be fast, reliable, and scalable. Your users will load dashboards in 1–2 seconds instead of 10–15 seconds. Your database will handle 10x more concurrent users. Your team will spend less time firefighting and more time building.
Start with the audit: measure your current performance. Identify the slowest dashboards. Optimise the database first. Configure caching. Then scale infrastructure. Monitor continuously.
If you need help, reach out. We have built and scaled Superset clusters for teams across Australia, the United States, and Canada. We know the gotchas, the patterns that work, and the infrastructure setup that scales.