topic Re: Old Query still included in data flow run in Dataflow

Old Query still included in data flow run

m_cherriman — Wed, 08 Nov 2023 15:07:33 GMT

I have a Dataflow which includes two queries one for products and one for mosaicdata. There used to be a third query called customers but I deleted it. However, when I run this Dataflow it fails (see fig 2) and it still references the customer query which seems to succeed at the initial level and then failes when it tries to write the data to a table, presumably because there is no data to pull back. Has anyone else encountered this issue?

fig 1

fig 2

Re: Old Query still included in data flow run

miguel — Wed, 08 Nov 2023 15:17:12 GMT

Hi!

If you try to refresh the dataflow once again, do you still get the same customers table showing up in the refresh history screen?

If yes, please reach out to the support team by creating a support ticket: https://support.fabric.microsoft.com/support

They'll primarily be looking for the .json file of your Dataflow to see if there are still references to the customers query in it.

If a discrepancy exists where the reference exists in the .json file but you do not see that query when you try to edit the dataflow, then they'll investigate further to see what could be the issue. However, if it happens that when you click on edit you still see the "customers" query then that might be a different issue that they'll be looking into as to why it might be happening and why you are not able to delete that query.

Re: Old Query still included in data flow run

m_cherriman — Wed, 08 Nov 2023 16:13:23 GMT

Thanks Miguel,

The Dataflow still fails and shows the customer query as failing. It's also referenced in the json file, but not visible in the query edit view. I'll raise a call and update the post when I get an answer.

Re: Old Query still included in data flow run

m_cherriman — Fri, 10 Nov 2023 07:33:55 GMT

After spending some time with one of the team at Microsoft we were able to resolve the issue by disabling the Enable Staging option for both queries. After doing this and publishing the data flow the Customer query was no longer in the json file and the query ran.

Hope this helps.

Re: Old Query still included in data flow run

miguel — Fri, 10 Nov 2023 16:52:14 GMT

Hey! If you raised a support ticket, would you mind sharing that support ticket here? We would love to better understand this case.

Re: Old Query still included in data flow run

mllopis — Mon, 13 Nov 2023 20:16:34 GMT

Hi m_cherriman - Kindly checking whether you got a chance to file a Support Ticket and you could share the ID with us.

We'd love to take a deeper look into this issue but require DataflowID and other telemetry information to do so, which we would get through the Support case.

Thanks,