How to Reduce Cloud Data Costs: A Practical Playbook for Data Teams

Cloud data costs have a way of sneaking up on organizations. In the early days, the bill is manageable and the ease of scaling feels like pure upside. Then the data team grows, more pipelines get added, more analysts run more queries, and suddenly you're staring at a $40,000/month Snowflake bill and no one is quite sure how it got there.

This is not a failure of cloud economics — it's a failure of cost architecture. The same scalability that makes cloud data platforms powerful also makes them easy to over-provision and inefficient to run without deliberate cost controls. The good news is that most organizations can cut their cloud data spend by 30–50% without touching query performance, using the levers in this guide.

Lever 1: Audit Who Is Spending What — Before You Optimize Anything

You cannot optimize what you don't measure. Before changing any configurations, run a query cost attribution report and find out which queries, users, and workloads are driving your bill. The distribution is almost always heavily skewed: 10–20% of queries typically account for 60–80% of compute spend.

In Snowflake, query history is in SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY:

SELECT
  user_name,
  warehouse_name,
  COUNT(*)                                    AS query_count,
  SUM(credits_used_cloud_services)           AS total_credits,
  AVG(execution_time) / 1000                 AS avg_seconds,
  SUM(bytes_scanned) / 1e9                   AS gb_scanned
FROM SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY
WHERE start_time >= DATEADD(day, -30, CURRENT_TIMESTAMP())
GROUP BY 1, 2
ORDER BY total_credits DESC
LIMIT 50;

In BigQuery, query costs are in INFORMATION_SCHEMA.JOBS_BY_PROJECT. In Redshift, use STL_QUERY joined with SVL_QUERY_METRICS.

What you're looking for: expensive queries running frequently, full-table scans on large tables, warehouse clusters idling between jobs, unused scheduled reports, and ad hoc analysts running unoptimized exploratory queries against production data.

Lever 2: Implement Auto-Suspend on Every Warehouse

This is the highest-ROI change most Snowflake customers can make, and it costs nothing in performance. By default, Snowflake warehouses remain running for 10 minutes after the last query completes. If your warehouse runs queries for 2 minutes per hour but is up for 60 minutes, you're paying for 58 minutes of idle compute.

Set auto-suspend to 60 seconds for development and ad hoc warehouses, and 5 minutes for production warehouses that have high query volume and benefit from keeping the cache warm:

ALTER WAREHOUSE dev_wh SET AUTO_SUSPEND = 60;       -- 1 minute
ALTER WAREHOUSE analytics_wh SET AUTO_SUSPEND = 300; -- 5 minutes
ALTER WAREHOUSE etl_wh SET AUTO_SUSPEND = 120;       -- 2 minutes

Combine this with appropriately sized warehouses. Most analytical queries that take 30+ seconds on an X-Small warehouse will complete in 5–10 seconds on a Small — but a Small costs 2× as much per credit. The crossover depends on your query concurrency and complexity. Run a benchmark before assuming bigger is cheaper.

In Redshift, the equivalent is pause/resume scheduling for clusters that only run during business hours. A cluster that runs 12 hours/day instead of 24 cuts its cost in half.

Lever 3: Partition and Cluster Your Tables

Cloud data warehouses charge for the amount of data scanned per query. A query that scans 10 TB costs roughly 100× more than one that scans 100 GB to answer the same question. Partitioning and clustering reduce scanned bytes by allowing the engine to skip irrelevant data entirely.

Partitioning divides a table into physical segments based on a column value — almost always a date or timestamp. Queries that filter on the partition column only scan the relevant partitions. If you have a 10 TB events table and most queries filter on the last 7 days, partitioning by date means those queries scan 7/365 of the data they would otherwise scan.

-- BigQuery: partition by event date
CREATE TABLE analytics.events
PARTITION BY DATE(event_timestamp)
CLUSTER BY user_id, event_type
AS SELECT * FROM analytics.events_raw;

-- Snowflake: cluster key on frequently filtered columns
ALTER TABLE analytics.events
  CLUSTER BY (event_date, user_id);

Clustering (Snowflake) and cluster BY (BigQuery) co-locate rows with the same values for a column on the same micro-partitions. Queries filtering on clustered columns skip entire micro-partitions without scanning them. This is most effective on columns with high cardinality that appear frequently in WHERE clauses — user_id, product_id, geography.

Typical impact: queries on well-partitioned and clustered tables scan 80–95% less data than on unoptimized tables. At BigQuery's $6.25/TB pricing, this translates directly to dollars.

Lever 4: Separate Your Workloads by Warehouse

Running all workloads on a single warehouse is convenient but expensive. ETL pipelines, analyst ad hoc queries, and BI tool refreshes have very different performance profiles and scheduling patterns — mixing them causes resource contention, forces larger warehouse sizes than any single workload needs, and makes it impossible to set appropriate auto-suspend policies.

A typical workload segmentation for a mid-size data team:

• etl_wh (Medium) — dbt runs, Fivetran syncs, pipeline jobs. Runs on a schedule, auto-suspend 2 minutes.
• analytics_wh (Small) — BI tool connections (Looker, Tableau, Mode). Always-on during business hours, suspended overnight and weekends.
• dev_wh (X-Small) — Individual analyst queries, notebook exploration. Auto-suspend 1 minute. Each analyst has their own virtual warehouse or shares this one.
• reporting_wh (Small) — Scheduled reports and dashboards. Runs during early morning refresh windows only.

This segmentation means you're right-sizing each workload rather than over-provisioning a single large warehouse to handle the worst-case scenario. In practice, moving from one Large shared warehouse to four appropriately sized segmented warehouses often cuts compute costs by 40–60%.

Lever 5: Kill Unused Pipelines and Dashboards

Every data organization has pipelines running that no one is using. They might be feeding dashboards that stakeholders stopped looking at six months ago. They might be populating tables that were created for a project that was canceled. They run every day, consuming compute and storage, and no one notices because the cost is diluted across the overall bill.

Run a usage audit:

• Pull dashboard view counts from your BI tool. Any dashboard not viewed in 60 days is a candidate for deprecation.
• Pull table access logs from your warehouse. Any table not queried in 90 days is a candidate for archiving.
• Pull pipeline run history from your orchestrator. Any pipeline that hasn't been accessed by a downstream consumer in 60 days should be evaluated.

Don't delete immediately — archive first. Move unused tables to cold storage tiers (Snowflake's STORAGE_INTEGRATION with S3 Glacier, BigQuery's long-term storage pricing). Pause unused pipelines rather than deleting them, in case someone complains. After 30 days of no complaints, delete.

In our experience working across dozens of data teams, deprecating unused infrastructure typically reduces storage costs by 20–35% and pipeline compute costs by 15–25%.

Lever 6: Use Incremental Processing Instead of Full Table Refreshes

Many data pipelines are written to process their entire dataset on every run: DROP TABLE; CREATE TABLE AS SELECT ... FROM source. This is simple to write and easy to reason about — and it gets exponentially more expensive as your data grows.

Incremental processing — where you only process new or changed records — reduces both compute time and cost by orders of magnitude once your tables reach meaningful scale. The mechanics differ by platform:

• dbt incremental models — Use is_incremental() to filter only new records. The merge strategy handles upserts automatically.
• Delta Lake MERGE — Supports upserts natively, updating existing rows and inserting new ones in a single atomic operation.
• BigQuery DML MERGE — Same concept in BigQuery's syntax, with partitioned tables to reduce scan cost.

-- dbt incremental model
{{ config(materialized='incremental', unique_key='order_id') }}

SELECT order_id, customer_id, total, status, updated_at
FROM {{ source('raw', 'orders') }}

{% if is_incremental() %}
  WHERE updated_at > (SELECT MAX(updated_at) FROM {{ this }})
{% endif %}

For a 1 billion row table, switching from a full refresh to an incremental load that processes 500,000 new records per day reduces the compute per run by roughly 99.95%. At scale, this is often the single highest-impact optimization available.

Lever 7: Implement Query Cost Controls and Spend Alerts

Reactive cost management — noticing the bill is high after the fact — is expensive. Proactive cost controls prevent overruns before they happen.

In Snowflake: Use resource monitors to set credit quotas per warehouse, per day or per month. When a warehouse hits its quota, queries are suspended automatically (or you receive an alert, depending on your configuration). This is especially important for development warehouses where individual analysts can accidentally run expensive full-table scans.

CREATE RESOURCE MONITOR dev_monthly_cap
  WITH CREDIT_QUOTA = 100
  FREQUENCY = MONTHLY
  START_TIMESTAMP = IMMEDIATELY
  TRIGGERS
    ON 75 PERCENT DO NOTIFY
    ON 100 PERCENT DO SUSPEND_IMMEDIATE;

In BigQuery: Set per-user and per-project byte quotas in IAM to cap how much any individual query can scan. Use BigQuery's cost controls in the project settings to set daily spend caps that halt queries when exceeded.

In Databricks: Use cluster policies to enforce auto-termination, restrict instance types, and set maximum runtime limits on job clusters. Use budgets in the Databricks account console to alert when monthly spend exceeds thresholds.

In all platforms, set up billing alerts at 50%, 75%, and 100% of your monthly budget so you have early warning before you exceed targets. Cloud provider billing alerts (AWS Budgets, GCP Budget Alerts, Azure Cost Management) provide a second layer of protection independent of the warehouse platform.

Lever 8: Right-Size Your Storage Tier

Storage is cheap relative to compute, but it's not free — and the way you store data affects your compute costs significantly. A few storage optimizations worth implementing:

• Use columnar formats. Parquet and ORC store data column-by-column rather than row-by-row. Analytical queries that only touch 5 of 50 columns scan roughly 10% of the data that row-format storage would require. If you're still storing raw data in CSV or JSON in S3, converting to Parquet can cut query costs by 60–80%.
• Apply lifecycle policies to cold data. Data older than 90 days is rarely queried interactively. Move it to cheaper storage tiers automatically: S3 Glacier Instant Retrieval, Azure Cool storage, GCS Nearline. For data you need to keep but almost never query, S3 Glacier Deep Archive costs roughly $0.00099/GB/month — about 23× cheaper than S3 Standard.
• Drop redundant copies. Data teams often have the same data in multiple places: in the source system, in an S3 raw layer, in a Bronze table, in a Silver table, in a legacy data mart, and in a BI tool's extract. Audit your data copies and consolidate where the duplicate doesn't serve a distinct purpose.

What 30–50% Savings Actually Looks Like

Across the organizations we've worked with, applying the levers above typically yields:

• Auto-suspend configuration: 15–25% reduction in compute
• Workload segmentation and right-sizing: 20–40% reduction in compute
• Incremental processing: 30–70% reduction in pipeline compute for mature datasets
• Partition and cluster optimization: 40–80% reduction in query compute for large tables
• Unused pipeline and dashboard deprecation: 15–30% reduction in overall spend

These savings are not additive — they overlap and interact. But the compounding effect of implementing several levers simultaneously consistently produces 30–50% overall reductions. For a team spending $20K/month, that's $6K–$10K/month recovered, which more than funds additional engineering capacity to do more with the savings.

When to Call in Outside Help

FinOps for data infrastructure is a specialized skill. The organizations that get the most out of these optimizations are those with a clear picture of their full cost topology — which workloads run when, which queries are most expensive, which pipelines have real downstream consumers versus which are orphaned. Building that picture takes time and experience.

If your data costs have grown faster than your data team's capacity to manage them — or if you've tried some of these levers and aren't seeing the expected results — an external architecture review can identify the highest-impact opportunities quickly. Our typical cost optimization engagement pays for itself in the first month of savings.