Modern Data Stack Guide: Snowflake, dbt, Fivetran and Databricks — What Each Layer Does and When You Need It

What is a modern data stack?

A composable, cloud-oriented data architecture in which separate tools handle ingestion, storage and processing, transformation, consumption and operations. It describes a pattern, not a product.

Older data platforms were often one large system that extracted, stored, transformed and reported. The modern pattern splits those responsibilities across specialized, usually cloud-hosted services connected through standard interfaces—most often SQL. The benefit is flexibility: each layer can be chosen, scaled and replaced independently. The cost is that someone has to design the whole, own the joins between layers and control spend across several subscriptions.

That trade-off is why the most useful question is not 'which modern data stack should we buy?' but 'which responsibilities does our workload actually require, and what is the simplest reliable way to cover each?'

What role does Snowflake play?

Snowflake is a cloud data platform most commonly used as an analytical warehouse: it stores structured and semi-structured data and runs SQL workloads for analytics and transformation.

Snowflake's documented architecture separates storage from compute. Data is held in a central storage layer, while independent compute clusters—called virtual warehouses—run queries. Conceptually, this means different teams or workloads can use separate compute without competing for the same resources, and compute can be sized or suspended independently of how much data is stored.

In practice Snowflake is often the place where raw data lands, where transformation runs (frequently orchestrated by dbt), and where BI tools query modelled data. It also supports non-SQL processing through Snowpark for Python, Java and Scala. Performance and cost depend heavily on warehouse sizing, query design and usage patterns, so any claim about either should be tested against your own workload rather than assumed.

What does dbt actually do?

dbt manages transformation as code. Teams write SQL (and, where supported, Python) models that dbt compiles and runs inside the data platform, with version control, tests, documentation and lineage built around them.

dbt does not extract data from source systems, and it does not store data itself. It works on data already in a warehouse or lakehouse, turning raw tables into staged, intermediate and business models. Its value is engineering discipline: each model is a file in Git, changes go through review, dependencies between models are explicit, tests such as unique, not_null and relationships run automatically, and documentation and lineage are generated from the same code.

That is why dbt is often described as bringing software-engineering practice to analytics. It is not mandatory. Well-governed SQL with its own testing and deployment can achieve similar outcomes; dbt simply provides a widely used, consistent framework for doing so.

What does Fivetran do?

Fivetran provides managed connectors that extract data from supported sources and load it into a destination such as a warehouse, keeping it synchronized on a schedule.

Its main benefit is not having to build and maintain every connector yourself—handling source APIs, schema changes and incremental syncs. For organizations with many SaaS sources, that can remove a substantial and recurring engineering burden.

• Connector availability: a source you depend on may not be supported, or may not expose every field you need
• Transformation is still required: landed data mirrors the source and must still be modelled and validated
• Costs and requirements vary: pricing is usage-based, so volume and sync frequency should be estimated before committing
• Alternatives exist: other managed connector services, cloud orchestration tools and custom ingestion can be the better fit for specific sources

When do you need Databricks?

Databricks is a data and AI platform built around Apache Spark for distributed processing. It is typically chosen for large-scale data engineering, complex transformations in PySpark or Scala, streaming, and machine-learning workloads.

Databricks describes its architecture as a lakehouse: data stored in open formats in cloud object storage, with governance and a SQL layer on top. It is therefore more than a warehouse competitor—its centre of gravity is programmatic data engineering and advanced analytics, while it also offers SQL warehouses for BI-style querying.

It becomes compelling when data is large, varied or semi-structured, when transformations are too complex or computationally heavy to express comfortably in SQL, or when data science and machine learning are core to the roadmap. For straightforward reporting across a handful of business systems, its capabilities may exceed what is needed.

Do I need Snowflake and Databricks?

Usually not both at the start. Choose based on the dominant workload. Both platforms now cover SQL analytics, data engineering and machine learning to a significant degree—the difference is where each typically starts strongest.

Organizations sometimes run both—for example, heavy processing in Databricks feeding modelled data into Snowflake for BI. That can be justified, but it doubles governance, skills and cost management, so it should follow a demonstrated need rather than a preference for completeness. Capabilities on both platforms change frequently; verify current features against official documentation before deciding.

How do these tools fit together?

Three patterns cover most situations. None is the only correct architecture.

What drives the cost of a modern data stack?

Usage-based pricing means cost follows behaviour. No meaningful figure can be given without knowing the workload, so estimate the drivers instead.

Many unexpected bills come from refresh frequencies set higher than decisions require, or from inefficient transformations re-processing full history every run. Both are design choices that can be reviewed.

• Data volume and growth rate
• Compute used for queries and transformations
• Storage, including retained history
• Connector volume—rows or records synchronized
• Refresh frequency: hourly syncs cost more than daily
• Transformation workloads and how efficiently they are written
• Development effort to build and change the platform
• Monitoring and incident handling
• Governance, security and access management

Common modern-data-stack mistakes

If the platform already exists and has become hard to change, the next article covers how to modernize it safely.

• Buying tools before defining requirements
• Ingesting everything instead of what decisions need
• No data ownership for key entities
• Weak transformation conventions and naming
• No automated testing
• Uncontrolled cost from oversized compute or over-frequent refresh
• Building dashboards before validating data
• Ignoring pipeline operations, alerting and on-call ownership