Forum Discussion
Solved
Dataflow Gen2 Fails to refresh - '_WriteToDataDestination' - 'Data source credentials are missing..
I have a Dataflow Gen 2 (CICD) item that unexpectedly stopped refreshing successfully. The dataflow collects data from several web sources, transforms it, and loads the data into Fabric Lakehouse Ta...
Hi dgeebs ,
Disabling “Enable staging” was a good tip, but in my case, the real issue came down to broken permissions and metadata bindings after deployment pipeline promotion.
- Reapplied Lakehouse permissions manually in the Prod workspace
→ Gave my user Build + Read/Write access again - Rebound the destination table in the Prod-stage Dataflow
→ Removed and re-added the Lakehouse output - Published and refreshed the dataflow worked successfully
After using pipelines, always verify Lakehouse permissions and rebind Dataflow destinations, even if everything looks fine.
Regards,
Akhil.- Reapplied Lakehouse permissions manually in the Prod workspace
Hi dgeebs,
If your destination is a Lakehouse, could you please check if "Enable staging" is checked for the queries in your Dataflow Gen2 CI/CD?
- If it is enabled, try unchecking it and then re-run the refresh. We've seen similar issues where staging being enabled causes credential-related failures during write operations to the Lakehouse.
- Even if it's enabled for just one query, it can cause refresh failures with errors related to missing or invalid credentials when writing to the Lakehouse.
Kindly uncheck staging for all queries and try refreshing again.
Let me know if that resolves the issue.