topic Gen2 Dataflow - Cannot turn off staging now? in Dataflow

Gen2 Dataflow - Cannot turn off staging now?

kblackburn — Thu, 14 Dec 2023 18:34:05 GMT

Hello. I have a few Gen2 dataflows that consume a csv file and load to Warehouse. I have had staging turned off. Yesterday, my flows started failing, and I saw that staging was turned on. Here is the message box that pops up when I attempt to disable staging. Is staging now mandatory with Gen2 flows landing in LH/WH? When staging is turned on I have multiple failures referencing fields that are not even in the dataset! Thanks for any feedback!

Re: Gen2 Dataflow - Cannot turn off staging now?

v-cboorla-msft — Fri, 15 Dec 2023 09:18:04 GMT

Hi kblackburn

Thanks for using Fabric Community.
At this time, we are reaching out to the internal team to get some help on this.
We will update you once we hear back from them.

Thanks

Re: Gen2 Dataflow - Cannot turn off staging now?

v-cboorla-msft — Fri, 15 Dec 2023 16:48:54 GMT

Hi kblackburn

Apologies for the inconvenience that you are facing here.

Enable staging is now required for the data warehouse destination. If you want to load without staging I’d recommend loading to the Lakehouse instead.

Thanks

Re: Gen2 Dataflow - Cannot turn off staging now?

kblackburn — Fri, 15 Dec 2023 18:22:44 GMT

Thank you v-cboorla-msft for the quick reply. I didn't see this info in the Nov or Dec release notes (perhaps I missed), but would be very helpful if it was mentioned!