What is dbt
dbt (data build tool) is an open-source framework created by Fishtown Analytics for executing, testing, and documenting SQL query. It simplifies and automates work with in-database transformation. dbt is distributed as an open-source product (without GUI) and as a cloud version with GUI.
Advantages
-
Clear Hierarchy for Starting the Calculation of Tables
dbt allows you to specify dependencies within each request. Based on these dependencies, the necessary tables are calculated in a row and a direct asynchronous graph is built.
-
Direct Ssynchronous Graph
This type of graph allows you to see exactly how the tables are formed. In dbt you can click on a specific table and see which tables it is being built from.
-
Jinja Templating Language Support
You need to write all the necessary calculations in the templates once – and the same model that worked on the old project will work on the new one. It saves a lot of time if you have a lot of repetitive tasks.
-
Ability to Test Data
When generating tables, you can test the data that appears in them, as well as set up sending notifications about detected errors.
-
Separation of Calculations Into Different Environments
If your project is at the testing stage, you can describe directly in the model that you need to work not with the entire data array, but with the selected piece.
Analytics Data Stack Structure
dbt Alternatives
-
Dataform
Dataform is an application to manage data in BigQuery, Snowflake, Redshift, and other data warehouses.
-
Apache Spark
Apache Spark is a framework that is used to process unstructured data in parallel. Spark supports Scala, Java, Python, R, and SQL programming languages.
-
Matillion
Matillion transforms businesses’data, across its various locations and forms, into cloud data warehouses to enable informed decision-making.
