Creating lakehouse using spark notebook

[AmirHeidari]

Hi,

We have created a new analytics platform which we mainly used Azure Synapse Analytics. However, recenly we are experimenting with microsoft fabric due to its pre-warmed up spark pools which could reduce our analytics reload time. Considering on migrating to fabric from synapse, I had some question regarding creating lakehouses.

In our current solution we are following medallion architecture. Data is coming through our bronze zones with simple copy activities.
In silver zone, we are utilizing spark to do transformations and business logic calculation. At this point, all the progress is stored in azure data lake Gen2.
When it comes to Gold zone, we are utilizaing spark to create databases and then the databases feed our BI tool (QlikSense). The reasoning is that we have various clients (Tenants) and we want to isolate their analytics database rather than having multi tenant architecture.

Database creation is done simply through spark notebook "Create Database if not exists". I cannot achieve a similar set up with fabric, if I assume my spark created database will be a fabric lakehouse at the end.

Such command will give error:

Is it possible to achieve a similar thing in Fabric through spark notebooks? What other option would you suggest?

[frithjof_v · Accepted solution]

I never tried it, but this blog post seems to suggest you can create a Lakehouse from a Notebook by using mssparkutils (beginning of article) or Semantic-Link (bottom of article):

https://fabric.guru/programmatically-creating-managing-lakehouses-in-fabric

[AndyDDC]

Hi @AmirHeidari you can't create a "database" in Fabric, you create a Lakehouse (or Warehouse). Even though they appear as databases that syntax won't work.

So you can either create a lakehouse in the gui or use the rest apis