Forum Discussion
Branched workspaces: Creating Views in the Datawarehouse that reference a Lakehouse table
- 1 year ago
Hi v-lgarikapat,
Thank you for your response and help. Since my initial post, we've tried a few scenarios and decided to house data stored in their own workspace (workspace dev 1), and have operational activities, like data pipelines and notebooks in another workspace (workspace dev 2). Still adopting the use of environments, but each environment will have two workspaces associated with it. Then, when devs pull code from workspace dev 1 only, they are only pulling operational artifacts. Using shortcuts proved confusing for Fabric when branching and pushing code to the develop branch, due to Lakehouses not being supported by git currently. We also didn't like the fact that the data in tables get deleted each time a table change is merged from a feature branch into develop.
Using this approach, we can:
* Create views and stored procedures in the data warehouse on data in the lakehouses, without having to use shortcuts.
* eliminated the issues we were encountering when branching code from the develop branch/ workspace into another branched workspace.
* still use version control and the PR process in git on the operational activity code.
* version control the data warehouse and SQL server db, by having it on another branch.
This is recent addition of VIEWS in Delta Lake. Now Databricks Delta Lake = Microsoft Fabric SQL