How to Use Dataflows Gen2 Without Creating a Lakehouse in Microsoft Fabric

[bdpr_95]

Hi, I have a use case where I want to create a Dataflow Gen2. However, I don’t want to store it in a Lakehouse, I want to use the same approach I have with Dataflows Gen1, where the purpose is to use it directly in a semantic model. How can I do this without Fabric automatically creating a staging Lakehouse for me?
Thanks for your help.

[v-hjannapu]

Hi @bdpr_95,

Thanks for reaching out to the Fabric community.

In Dataflow Gen2, this is expected behavior. Unlike Gen1, Gen2 always needs a place to store the data while it is processing. That’s why Fabric automatically creates a staging Lakehouse if you don’t select one. As of now, there is no option to skip this Lakehouse step in Gen2.

If your requirement is to use dataflow only for transformation and directly use it in a semantic model without storing data, then Dataflow Gen1 is the right choice.

Otherwise, in Gen2 you can use the data from the Lakehouse output for your semantic model.

Kindly refer to the below documentation links for better understanding:
 Data in Dataflow Gen2 staging - Microsoft Fabric | Microsoft Learn

Hope the above provided information help you resolve the issue, if you have any further concerns or queries, please feel free to reach out to us.
Regards,
Community Support Team.

[bdpr_95]

the problem here is, I have 2 environments, dev and prod. to do the deployment to previous stage (via deployment pipeline feature) dataflows gen1 cannot be moved, only dataflows gen2.

[v-hjannapu · Accepted solution]

Hi @bdpr_95 ,
Thanks for the details.
Yes, you are correct Since deployment pipelines support only Dataflow Gen2, you will need to go with Gen2 for Dev to Prod movement. But in Gen2, the staging Lakehouse is required and we cannot avoid it at the moment. Even if you only want to use the data in a semantic model, Fabric will still create or use a Lakehouse in the backend. So in your case, the best way is to use Gen2, let it load data into a Lakehouse you can keep it just for staging purpose, and then connect your semantic model to it. As of now, there is no option to achieve Gen1-like behavior without storage in Gen2.

Hope this helps. Let me know if you need anything else.
Regards,
Community Support Team.

[v-hjannapu]

Hi @bdpr_95,
I hope the information provided above assists you in resolving the issue. If you have any additional questions or concerns, please do not hesitate to contact us. We are here to support you and will be happy to help with any further assistance you may need.

Regards,
Community Support Team.

[v-hjannapu]

Hi @bdpr_95,
I hope the above details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We are always here to support you.


Regards,
Community Support Team.