Forum Discussion
Branched workspaces: Creating Views in the Datawarehouse that reference a Lakehouse table
- 1 year ago
Hi v-lgarikapat,
Thank you for your response and help. Since my initial post, we've tried a few scenarios and decided to house data stored in their own workspace (workspace dev 1), and have operational activities, like data pipelines and notebooks in another workspace (workspace dev 2). Still adopting the use of environments, but each environment will have two workspaces associated with it. Then, when devs pull code from workspace dev 1 only, they are only pulling operational artifacts. Using shortcuts proved confusing for Fabric when branching and pushing code to the develop branch, due to Lakehouses not being supported by git currently. We also didn't like the fact that the data in tables get deleted each time a table change is merged from a feature branch into develop.
Using this approach, we can:
* Create views and stored procedures in the data warehouse on data in the lakehouses, without having to use shortcuts.
* eliminated the issues we were encountering when branching code from the develop branch/ workspace into another branched workspace.
* still use version control and the PR process in git on the operational activity code.
* version control the data warehouse and SQL server db, by having it on another branch.
Hi caseybks
There are views in Fabric DataWarehouse. you can find Tables only that are generated from data lake ( apache spark ) and delta lake. You can connect with me in Linkedin ( https://www.linkedin.com/in/bhavesh-patel-472829106/ ) and I will go through each and individual steps of Delta Lake. Cheers
This is what we have done so far and it is successful outcome project.
Bronze Tables -- > Silver Tables -- > Gold Tables. All these is happening in In memory engine of Delta Lake.
- caseybks1 year agoAdvocate III
Hi BhaveshPatel, thanks for your reply, but you can create Views in Fabric datawarehouse.
A simple example is:
CREAT VIEW datawarehouse.dbo.view
AS
SELECT* FROM lakehouse.gold.table1