# dbt-fivetran-marketing
dbt Core transformations for Marketing data ingested by Fivetran and used by Fansifter.


### Visualising DBT models
DBT provides browser based UI for viewing resource dependencies. You can also view model descriptions etc.

* First step is to create up to date catalog.
```bash
dbt docs generate
```

* Then you can serve catalog.
```bash
dbt docs serve # by default opens web page at http://localhost:8080/#!/overview
```
_Read more from here:_ https://docs.getdbt.com/reference/commands/cmd-docs


### Introduction to DBT commands
<b>Here are somme commands with explanations that help to run different
parts of this project.</b>

Usually it's good practice to update seeds table when continuing working on DBT repository:
```bash
dbt seed # this will update all models located in seeds folder
```
_Read more from here:_ https://docs.getdbt.com/docs/build/seeds
<br>
_In some cases the structure for seed table has changed, then it maybe be rquired to
drop the table directly from database and then rerunning seed command._



<b>3 main commmands for running models are:</b>
* DBT RUN - runs selected resources
* DBT TEST - runs tests for selected resources
* DBT BUILD - executes both run and test commands on selected resources


<b>Selecting or excluding resources(models, tests, tags, folders) for execution:</b>
_Tags are defined in the dbt_project.yml_
* DBT RUN --select "MODEL"
* DBT RUN --exclude "MODEL"

```bash
dbt run --select "INT_FACEBOOK_ADS__AD_VIEWS_DBT"
dbt run --select models/facebook_ads/intermediate
dbt run --select models/core --exclude "INT_FACEBOOK_ADS__AD_VIEWS_DBT"

dbt test --select "INT_FACEBOOK_ADS__AD_VIEWS_DBT"

dbt build --select "INT_FACEBOOK_ADS__AD_VIEWS_DBT"
``` 
_Read more from here:_ https://docs.getdbt.com/reference/node-selection/syntax

<b>Specifying more complex selections. Use can use graph operators to run
either descendants or ancestors models:</b>
```bash
dbt run --select "INT_FACEBOOK_ADS__AD_VIEWS_DBT+" # select model and all descendants
dbt run --select "+INT_FACEBOOK_ADS__AD_VIEWS_DBT" # select model and all ancestors

``` 
_Read more from here:_ https://docs.getdbt.com/reference/node-selection/graph-operators <br>
https://docs.getdbt.com/reference/node-selection/set-operators#unions <br>
Yaml selector provides options to write complex select queries: https://docs.getdbt.com/reference/node-selection/yaml-selectors

<b>DBT retry command </b><br>
re-executes the last dbt command from the node point of failure. If the previously executed dbt command was successful, retry will finish as no operation. 
```bash
dbt retry
``` 
_Read more from here:_ https://docs.getdbt.com/reference/commands/retry

<b>Troubleshooting with the ls command.</b><br>
Constructing and debugging your selection syntax can be challenging. To get a "preview" of what will be selected, we recommend using the list command. 
```bash
dbt ls --select models/facebook_ads/intermediate
dbt ls --select "INT_FACEBOOK_ADS__AD_VIEWS_DBT"
``` 
_Read more from here:_ https://docs.getdbt.com/reference/node-selection/syntax#troubleshoot-with-the-ls-command

<b>DBT full refresh.</b><br>
If you provide the --full-refresh flag to dbt run, dbt will treat incremental models as table models. This is useful when
* The schema of an incremental model changes and you need to recreate it.
* You want to reprocess the entirety of the incremental model because of new logic in the model code.
```bash
dbt run --full-refresh
``` 
_Read more from here:_ https://docs.getdbt.com/reference/commands/run#refresh-incremental-models

<b>Generating models.yml file based on existing SQL.</b><br>
You can use dbt-codegen package to generate basic yaml file matching the provided .sql models
```bash
dbt --quiet run-operation generate_model_yaml --args '{"model_names": ["MODEL_1", "MODEL_2", ....]}' > models/[YOUR PATH]/models.yml
```
_Read more from here:_ https://github.com/dbt-labs/dbt-codegen/blob/main/README.md

<b>Additional resources:</b><br>
The Ultimate Guide to DBT- https://count.co/canvas/JpkaYdqr9oN