Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Problem with Query validation in Dataflows

I began experiencing an issue with dataflows where I am no longer able to save queries, I get the validating message for more than 3 hours and it doesn't crash or advance. Previously it was taking about 3 minutes.

image.png

 

I already tried running a similar query in the desktop app and it runs really fast so I don't think it's the complexity of the query (also it was working fine last week)

 

Has anyone encountered this issue before?

 

Status: New
Comments
dyabes
Helper I

Have the same problem but these are new queries. Just going to follow this conversation.

v-qiuyu-msft
Community Support

Hi all, 

 

Please test again. If the issue still occurs, I would suggest you create a support ticket to get help. 

 

Support Ticket.gif

 

Best Regards,
Qiuyun Yu

LABORSAL
Advocate II

So the only solution I found was leaving it overnight and it evenutally finishes 😕

Anonymous
Not applicable

We are still having issues with this.  For our Use Case, we just needed to switch the underlying source from Sharepoint Online folder to a Gateway folder entry.  Change was super straight forward, but when we tried to save, the validating queries never returned (think it actually times out after 24 hours).  

The one hack we discovered was to export the JSON, edit the necessary lines and import it back in. Then we just went to the DataFlow settings to bind the Gateway connection.  We tried editing the DataFlow afterwards to see if it still gets stuck in infinite validing queries and surely enough it did.  This is certainly a bug that Microsoft needs to look into... 

pat_mecee
Frequent Visitor

Following thread

camvt2014
Frequent Visitor

Following thread

Alexander76877
Helper II

Hi, I also experienced this problem. In fact, I think it is a bug (like others have commented) and I found a solution to it which at least worked for me.

 

  • My dataflow contains a couple of entities, some reading from a file, some from a folder with few files, some from a SQL server. Some merges but not too complicated
  • When editing the queries, some entities in my dataflow take 1-2minutes to show the table but that´s ok due to the transformations applied.
  • When clicking on "save & close" it took 60min if I was lucky, had timeout, or it was running until next day.
  • Refreshing the dataflow took only 2min, though.
  • I was *that* close to opening a ticket with MS
  • As a last resort it tried this:
    • create a new blank workspace or create a new dataflow within the same workspace
    • copy paste the entities from the bugged dataflow
    • save&close now takes <2min

All datasets referring to this dataflow must be redirected as the dataflow ID changed but at least it works now. Seems the bug is not copied when copying the queries.

Alexander

Daryl-Lynch-Bzy
Resident Rockstar

I agree with @LABORSAL.  The performance of the Validating Output Schema when saving changes to a Dataflow is too slow.  


There is no correlation between Dataflow refresh and Validation upon save and close.

 

I can wait up to an hour for the Validation upon saving to finish.  When done, the refresh takes less than 15 secounds.

pdbenbow
Resolver II

Looks like background validation for dataflows is a planned feature scheduled for release later this month, so maybe this will become less of a problem soon: Background validation for dataflows - Power Platform Release Plan | Microsoft Docs

LABORSAL
Advocate II

Hopefully @pdbenbow's finding will be a solution, this keeps happening and if your data is very large it reaches a "time out" and simply doesn't save it (I've had the issue happen, I make smaller data source and it saves, then run it with new data and no problem)