Forum Discussion
My DataFlow Gen 2 Fails with Unknown Evaluation Error 999999 (Request ID: fcd89914-0dd9-44ea-9b09-94
I do not understand why the Dataflow gen 2 is too buggy. This should be a revolutionary product from Fabric but it just cannot work without errors all the time.
I have a dataflow gen 2 that takes data from a lakehouse and transforms it and saves it in a destination. It has been working very fine for some few days, now it does not work baut instead shows this error. Error Details: We encountered an error during evaluation. Details: Unknown evaluation error code: 999999 (Request ID: fcd89914-0dd9-44ea-9b09-940f84fa6e3e).
The erroe message is very vague. Yes, I have access to the undelying data. The lakehouse it connects to has shortcuts from mirroring SQL Server. I do have access to this shortcut lakehouse data.
Can any one help me with this ?
11 Replies
AlvenMicrosoft Employee
Hi sholy29,
from a quick look to the telemetry for the Request ID you shared, I see that some of your queries with staging enabled got a problem which seems linked to a Microsoft Entra ID token: if the error persists, please raise a support ticket.
Worth mentioning that, given your dataflow is writing to destinations, I assume that you do not need to stage such queries: if that's the case, you can try to uncheck the "Enable staging" option (see picture below) for all of them and re-run the dataflow by clicking "Save & run"
Best regards,
Alessandro
Thanks Alessandro.
I do have many queries that are not writing to a destination. I need those queries because I need to merge (join) them (denormalise) with other queries to have the final ouput table that then writes to a destination. I noticed that it is those queries without a destination that has staging enabled by default.
Could this be the cause of the problem? Should I disable staging for the queries without destination?
Thank you
- Alven
Hi sholy29,
staging such queries could be useful in some cases for performance reasons (see Data in Dataflow Gen2 staging - Microsoft Fabric | Microsoft Learn and Best practices for creating a dimensional model using dataflows - Power Query | Microsoft Learn), but it really depends on the kind of transformations and how big the datasets of such queries are.
The issue you reported should not happen, and if it persists consider raising a support ticket. But, as a workaround, you can uncheck the "enable staging" option for such queries and see if
- dataflow refresh succeeds
- the running time of the dataflow refresh does not increase significantly because the dataflow does not use staging anymore
Best regards,
Alessandro
Hi Alessandro,
Thank you for your quick response, that solved the problem. Thank you very much!
Does that mean I can't use the function at all, even though it should offer advantages? Or do I have to make special settings for it? Can you help me with that?
- Alven
Hi CRadziwill ,
usage (or not) of staging should not cause any failure, and if you instead get one, it usually means there is a problem somewhere thay requires an investigation / support ticket.
In general, staging can help with performance in some cases, but it really depends on the scenario.
For example, if your query does not perform complex transformations and is simply copying data from a SQL database to a Lakehouse using an output destination, staging is not needed, and it's actually disabled automatically when setting the output destination.
Some useful information can be found in Best practices for getting the best performance with Dataflow Gen2 in Fabric Data Factory - Microsoft Fabric | Microsoft Learn.
Regards,
Alessandro
v-prasareCommunity Support
Hi sholy29 ,
Thanks for reaching MS Fabric community supportIn this case please consider raising a support ticket for further assistance. To raise a support ticket with request Id and other details for Fabric and Power BI, kindly follow the steps outlined in the following guide:
How to create a Fabric and Power BI Support ticket - Power BI | Microsoft Learn
Thanks,
Prashanth Are
- v-prasare
This issue has been acknowledged and is now listed in the Known Issues Tracker under the title:
"Refreshes running slower or timing out for Dataflows Gen 2.0 with CI/CD."As the issue is already known and being monitored, we are closing this conversation since no further action is required at this point. For any new queries or updates, please feel free to raise a new thread in the Fabric Community Forum: https://community.fabric.microsoft.com/
Thanks again for your participation in the Fabric Community Forum!
