Forum Discussion
Organising ETL Best Practice
Hi all I'm building a new data model from 3 dataflows. The dataflows ingest Snowflake data and all pull in the same table structure from 3 different entities. I removed all the unnecessary ta...
To try and explain a little more clearly, here's a picture:
Right now I'm confident in the Yellow to Blue step. But it feels like I'm combining the Blue to Orange and the Orange to Pink steps in a way that's going to cause trouble later on by using the blue tables throughout the process. Should I leave the blue tables untouched (and just deselect "Enable Load" on them), then duplicate out new tables for the orange steps with the pink step just being their final form, or is it OK to just build it all into the tables created in the blue step and trust that future me (or future colleague) will be able to unpick it all?