topic Re: Many steps were not saved in Dataflow

Many steps were not saved

ebjim — Mon, 23 Oct 2023 13:30:44 GMT

I have been observing that if I save a dataflow by clicking the arrow in the Publish button and choosing Publish Later, I lose at least half the steps that I created prior to saving. This happens to larger, more complex dataflows, i.e a DF with 2 tables and 6 steps total won't be affected.

Has anyone else experienced this issue? Is there a right way to save and not publish?

Re: Many steps were not saved

v-cboorla-msft — Mon, 23 Oct 2023 18:29:08 GMT

Hi ebjim

Thanks for using Fabric Community.

I have reached the internal team for help on this. I will update you once I hear from them.
Appreciate your patience.

Re: Many steps were not saved

miguel — Tue, 24 Oct 2023 14:25:54 GMT

Do you happen to have a way in which we could replicate this behavior?

Re: Many steps were not saved

ebjim — Tue, 24 Oct 2023 14:29:49 GMT

miguel I can do a screenshare in Teams. Just reach out.

Re: Many steps were not saved

ebjim — Tue, 24 Oct 2023 14:30:06 GMT

v-cboorla-msft Thank you!

Re: Many steps were not saved

miguel — Tue, 24 Oct 2023 14:33:14 GMT

would you mind sharing the steps here? Will then share this thread with the engineers so they can reproduce this issue on their environments and determine the root cause.

Re: Many steps were not saved

ebjim — Tue, 24 Oct 2023 14:45:17 GMT

This example shows me adding a column name to a Table.Distinct method. I save by choosing Publish Later. I then re-open the dataflow and that column name is not there. I have had instances where multiple steps were not saved.

Re: Many steps were not saved

ebjim — Fri, 27 Oct 2023 19:07:10 GMT

miguel FYI, this issue is still occurring as of 15 minutes ago.

Re: Many steps were not saved

miguel — Fri, 27 Oct 2023 19:12:02 GMT

We believe that we've determined the issue and we're working on a way to address it.

The issue is related to how frequently and how successfully the draft of a particular dataflow is being saved. This means that in some cases the user actions might be a bit too fast for the Dataflow service to catch up before publishing.

Re: Many steps were not saved

ebjim — Fri, 27 Oct 2023 20:22:28 GMT

Good to know that a fix is on its way, but it would be good to know: how fast is too fast?

Re: Many steps were not saved

miguel — Sat, 28 Oct 2023 02:34:12 GMT

Good question! It could be a timing situation where you lost connection during a particular moment and the draft couldn't be saved and then you published an earlier version of the dataflow. Since there are many scenarios in which this could happen, beyond just the ones previously mentioned, we will address this by giving you a notification on the status of the autosave similarly to how it's done in Microsoft Office apps telling you when the last autosave was made and if there are any issues