SQL Optimization Playbook for Warehouse Analysts

Modern cloud warehouses give you sophisticated tuning knobs, but documentation often lags behind. This guide shows how to annotate SQL plans, highlight critical snippets, and attach performance artifacts so every reviewer can reproduce improvements.

Saved articles
Hides the site's navigation and the panels around the article; press Escape to leave.

Online at https://diogoribeiro7.github.io/analytics-blog-jekyll/2024/04/05/sql-optimization-guide/

Topics

Modern cloud warehouses give you sophisticated tuning knobs, but documentation often lags behind. This guide shows how to annotate SQL plans, highlight critical snippets, and attach performance artifacts so every reviewer can reproduce improvements.

Baseline query

1
2
3
4
5
6
7
8
SELECT
  customer_id,
  SUM(spend) AS total_spend,
  AVG(spend) AS avg_spend,
  ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY purchase_ts DESC) AS purchase_rank
FROM analytics.fact_orders
WHERE purchase_ts >= DATEADD('day', -90, CURRENT_DATE)
GROUP BY 1

Use the built-in explain tools to gather diagnostics:

1
2
3
4
5
6
7
8
9
10
11
12
13
EXPLAIN USING JSON
SELECT *
FROM (
  -- baseline aggregation
  SELECT
    customer_id,
    SUM(spend) AS total_spend,
    AVG(spend) AS avg_spend
  FROM analytics.fact_orders
  WHERE purchase_ts >= DATEADD('day', -90, CURRENT_DATE)
  GROUP BY 1
) src
JOIN analytics.dim_customer dc USING (customer_id);

Optimization checklist

  1. Materialize the 90-day window as an incremental model.
  2. Cluster the fact table on purchase_ts to prune partitions.
  3. Replace repeated JSON parsing with persisted staged columns.
  4. Cache high-cardinality dimension joins using search optimization services.

Note: Store the JSON explain output in _datasets/ so reviewers can diff the query plan over time.

Annotate performance wins

Change Before (s) After (s) Impact
Clustering on purchase_ts 22.4 9.7 2.3× faster
Incremental materialization 9.7 4.5 2.1× faster
Persisted JSON attributes 4.5 3.8 1.2× faster

Wrap up by linking to dbt models, scheduling notes, and alert thresholds so stakeholders can keep the warehouse humming.

Your private highlights

Kept in this browser only and never sent anywhere. These are your notes, not comments. With text selected, Alt+Shift+H highlights it and Alt+Shift+N adds a note.

Select a passage of the article to highlight it.

    Your reading data

    Embed interactive plots, widgets, and demos using <figure>, <iframe>, or <div class="interactive-embed"> containers. Ensure each embed includes descriptive captions for accessibility.

    © 2024 Diogo Ribeiro. Text and figures under CC BY 4.0.

    How to cite

    Use the quick export buttons to save citations for reference managers or copy the formatted text directly.

    Diogo Ribeiro (2024). SQL Optimization Playbook for Warehouse Analysts. DataLog | Data Science & Research Theme. https://diogoribeiro7.github.io/analytics-blog-jekyll/2024/04/05/sql-optimization-guide/.

    BibTeX

    RIS

    EndNote

    Open science & reproducibility badges

    These badges highlight the transparency practices applied to this work. Hover or focus on each badge to learn more about the criteria.

    • Open Data Dataset and code repository published with permissive license. Public repository, DOI issued, README with reproduction steps.
    • Reproducible Workflow Containerized environment and automated tests provided. Continuous integration pipeline with reproducibility checks.
    • Transparent Peer Review Peer review reports archived with DOI and linked to article. Open peer review statement and archived reports on Zenodo.

    Related posts

    Loading mathematical content