Forum Discussion

coverturfLSH's avatar
coverturfLSH
Resolver II
1 year ago
Solved

Dataflow refresh not running

Hi,   I have a Dataflow Gen2 that had been running fine for many months now. Today I had an issue where the Dataflow was taking much longer than usual (+2 hours vs normal 3 minute run). I canceled ...
Dataflow
  • coverturfLSH's avatar
    coverturfLSH
    1 year ago

    Interim (final?) solution was to export the template (PQT) for the old not working, import the template in a new DFG2, and then set a default data destination. Made it fairly seemless. 

v-veshwara-msft's avatar
v-veshwara-msft
Community Support
1 year ago

Hi coverturfLSH ,

Thanks for reaching out to Microsoft Fabric Community.

Thanks for confirming the interim solution using the Power Query Template and setting the default data destination, that’s a practical and effective approach when a Dataflow becomes unresponsive or fails to refresh as expected.

Also, thank you miguel  for outlining some of the key improvements available in the Dataflow Gen2 CI/CD experience.

Here are some notable benefits of Dataflow Gen2 with CI/CD and platform improvements that you might still find useful:

  1. Multitasking support: Open multiple Dataflow Gen2 items alongside other Fabric items, improving your overall authoring experience.

  2. Workspace folder support: Create dataflows directly in structured workspace folders for better organization.

  3. Deployment pipelines: Automate movement of dataflows between environments (e.g Dev to Test to Prod) using deployment pipelines.

  4. Fabric settings and scheduler: Manage refresh schedules and Dataflow Gen2 settings directly within Fabric.

  5. Public APIs (preview): Automate and manage Dataflow Gen2 operations including create, update, delete, scheduling, and monitoring.

The Public APIs for Dataflow Gen2 enable advanced scenarios even without Git integration, such as:

  1. Automated deployment across environments

  2. Monitoring and alerts for tracking failures or performance issues

  3. Integrating data from multiple sources into unified workflows

  4. Custom error handling and retry mechanisms.

Here are some links which you might find useful:
Dataflow Gen2 CI/CD, GIT integration and Public APIs (Generally Available) | Microsoft Fabric Blog | Microsoft Fabric

Dataflow Gen2 with CI/CD and Git integration - Microsoft Fabric | Microsoft Learn

 

Hope this helps. Please reach out for further assistance.

If you’ve already raised a support ticket, please consider sharing any insights or resolution steps provided by the support team here, so others in the community facing similar issues can benefit.

 

Please consider marking the helpful reply as Accepted Solution and giving kudos to assist others with similar issues.

Thnak you.