Problem with Query validation in Dataflows

[LABORSAL]

I began experiencing an issue with dataflows where I am no longer able to save queries, I get the validating message for more than 3 hours and it doesn't crash or advance. Previously it was taking about 3 minutes.

I already tried running a similar query in the desktop app and it runs really fast so I don't think it's the complexity of the query (also it was working fine last week)

Has anyone encountered this issue before?

[Anonymous]

Have the same problem but these are new queries. Just going to follow this conversation.

[v-qiuyu-msft]

Hi all, 

Please test again. If the issue still occurs, I would suggest you create a support ticket to get help. 

Best Regards,
Qiuyun Yu

[LABORSAL]

So the only solution I found was leaving it overnight and it evenutally finishes 😕

[Anonymous]

We are still having issues with this.  For our Use Case, we just needed to switch the underlying source from Sharepoint Online folder to a Gateway folder entry.  Change was super straight forward, but when we tried to save, the validating queries never returned (think it actually times out after 24 hours).  

The one hack we discovered was to export the JSON, edit the necessary lines and import it back in. Then we just went to the DataFlow settings to bind the Gateway connection.  We tried editing the DataFlow afterwards to see if it still gets stuck in infinite validing queries and surely enough it did.  This is certainly a bug that Microsoft needs to look into... 

[pat_mecee]

Following thread

[Anonymous]

Following thread

[Anonymous]

Hi, I also experienced this problem. In fact, I think it is a bug (like others have commented) and I found a solution to it which at least worked for me.

• My dataflow contains a couple of entities, some reading from a file, some from a folder with few files, some from a SQL server. Some merges but not too complicated
• When editing the queries, some entities in my dataflow take 1-2minutes to show the table but that´s ok due to the transformations applied.
• When clicking on "save & close" it took 60min if I was lucky, had timeout, or it was running until next day.
• Refreshing the dataflow took only 2min, though.
• I was *that* close to opening a ticket with MS
• As a last resort it tried this:
  • create a new blank workspace or create a new dataflow within the same workspace
  • copy paste the entities from the bugged dataflow
  • save&close now takes <2min

All datasets referring to this dataflow must be redirected as the dataflow ID changed but at least it works now. Seems the bug is not copied when copying the queries.

Alexander

[Daryl-Lynch-Bzy]

I agree with @LABORSAL.  The performance of the Validating Output Schema when saving changes to a Dataflow is too slow.  

There is no correlation between Dataflow refresh and Validation upon save and close.

I can wait up to an hour for the Validation upon saving to finish.  When done, the refresh takes less than 15 secounds.

[pdbenbow]

Looks like background validation for dataflows is a planned feature scheduled for release later this month, so maybe this will become less of a problem soon: Background validation for dataflows - Power Platform Release Plan | Microsoft Docs

[LABORSAL]

Hopefully @pdbenbow's finding will be a solution, this keeps happening and if your data is very large it reaches a "time out" and simply doesn't save it (I've had the issue happen, I make smaller data source and it saves, then run it with new data and no problem)