Forum Discussion

Asmatte's avatar
Asmatte
New Member
1 year ago
Solved

Fabric Dataflow Gen2 – Refresh succeeds despite Power Query error

Hi,   I’ve built a Data Pipeline that refreshes a Dataflow Gen2 and logs the result of this refresh into a log Delta table.   In this pipeline, I’ve added an if condition that checks the status o...
  • miguel's avatar
    miguel
    1 year ago

    When you do the "save & run", if the save operation fails then you'll be able to check it through the "Check validation". If it fails, it means that the version that you tried to save didn't actually save, so you are using the previous version of your Dataflow.

     

    You have a couple of ways now to check what your published Dataflow looks like. Here are a the list of ways to check them out from easiest to more complex:

    1. Opening the Dataflow: When you use Dataflow Gen2 with CI/CD capabilities, you can always discard any previously unsaved versions of your Dataflow and then open again the Dataflow which should match the version of the Dataflow that the pipeline can trigger for refresh
    2. Checking Git: As this is not an option for you, I'll just skip this one but its also one of the easiest ones where you could just check Git and see exactly what the M code looks like
    3. REST API: This is a bit more complex, but you can leverage the REST API endpoint for GET Dataflow definition. The actual M code for your is in the path: mashup.pq and you'll need to decode the payload which is in base64 to then see your full Dataflow mashup script.

     

    In other words, if we find any issues with your Dataflow during the "save" operation, then it will simply not be committed whatsoever.

     

    If the intent is to trigger an error to see how things look like in Dataflows when an error occurs, you could take a different path where you can create a Fabric item (like a Lakehouse), create a Dataflow that connects to a table, save and run the Dataflow, then rename the table in the lakehouse and try to run the Dataflow again. It should fail as you've changed the name of the table in the Fabric item after the Dataflow was correctly saved and no validations failed.