Resume where you left off
Online at https://diogoribeiro7.github.io/analytics-blog-jekyll/2024/04/05/sql-optimization-guide/
Topics
Modern cloud warehouses give you sophisticated tuning knobs, but documentation often lags behind. This guide shows how to annotate SQL plans, highlight critical snippets, and attach performance artifacts so every reviewer can reproduce improvements.
Baseline query
1
2
3
4
5
6
7
8
SELECT
customer_id,
SUM(spend) AS total_spend,
AVG(spend) AS avg_spend,
ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY purchase_ts DESC) AS purchase_rank
FROM analytics.fact_orders
WHERE purchase_ts >= DATEADD('day', -90, CURRENT_DATE)
GROUP BY 1
Use the built-in explain tools to gather diagnostics:
1
2
3
4
5
6
7
8
9
10
11
12
13
EXPLAIN USING JSON
SELECT *
FROM (
-- baseline aggregation
SELECT
customer_id,
SUM(spend) AS total_spend,
AVG(spend) AS avg_spend
FROM analytics.fact_orders
WHERE purchase_ts >= DATEADD('day', -90, CURRENT_DATE)
GROUP BY 1
) src
JOIN analytics.dim_customer dc USING (customer_id);
Optimization checklist
- Materialize the 90-day window as an incremental model.
- Cluster the fact table on
purchase_tsto prune partitions. - Replace repeated JSON parsing with persisted staged columns.
- Cache high-cardinality dimension joins using search optimization services.
Note: Store the JSON explain output in
_datasets/so reviewers can diff the query plan over time.
Annotate performance wins
| Change | Before (s) | After (s) | Impact |
|---|---|---|---|
Clustering on purchase_ts |
22.4 | 9.7 | 2.3× faster |
| Incremental materialization | 9.7 | 4.5 | 2.1× faster |
| Persisted JSON attributes | 4.5 | 3.8 | 1.2× faster |
Wrap up by linking to dbt models, scheduling notes, and alert thresholds so stakeholders can keep the warehouse humming.
Your private highlights
Kept in this browser only and never sent anywhere. These are your notes, not comments. With text selected, Alt+Shift+H highlights it and Alt+Shift+N adds a note.
This browser is not letting the site keep data, so saving, progress and highlights are off.
Select a passage of the article to highlight it.
Your reading data
Embed interactive plots, widgets, and demos using <figure>, <iframe>, or <div class="interactive-embed"> containers. Ensure each embed includes descriptive captions for accessibility.
© 2024 Diogo Ribeiro. Text and figures under CC BY 4.0.
How to cite
Use the quick export buttons to save citations for reference managers or copy the formatted text directly.
Diogo Ribeiro (2024). SQL Optimization Playbook for Warehouse Analysts. DataLog | Data Science & Research Theme. https://diogoribeiro7.github.io/analytics-blog-jekyll/2024/04/05/sql-optimization-guide/.
BibTeX
RIS
EndNote
Open science & reproducibility badges
These badges highlight the transparency practices applied to this work. Hover or focus on each badge to learn more about the criteria.
- Open Data Dataset and code repository published with permissive license. Public repository, DOI issued, README with reproduction steps.
- Reproducible Workflow Containerized environment and automated tests provided. Continuous integration pipeline with reproducibility checks.
- Transparent Peer Review Peer review reports archived with DOI and linked to article. Open peer review statement and archived reports on Zenodo.