Forum Discussion
Fabric Dataflow Gen2 – Refresh succeeds despite Power Query error
When you do the "save & run", if the save operation fails then you'll be able to check it through the "Check validation". If it fails, it means that the version that you tried to save didn't actually save, so you are using the previous version of your Dataflow.
You have a couple of ways now to check what your published Dataflow looks like. Here are a the list of ways to check them out from easiest to more complex:
- Opening the Dataflow: When you use Dataflow Gen2 with CI/CD capabilities, you can always discard any previously unsaved versions of your Dataflow and then open again the Dataflow which should match the version of the Dataflow that the pipeline can trigger for refresh
- Checking Git: As this is not an option for you, I'll just skip this one but its also one of the easiest ones where you could just check Git and see exactly what the M code looks like
- REST API: This is a bit more complex, but you can leverage the REST API endpoint for GET Dataflow definition. The actual M code for your is in the path: mashup.pq and you'll need to decode the payload which is in base64 to then see your full Dataflow mashup script.
In other words, if we find any issues with your Dataflow during the "save" operation, then it will simply not be committed whatsoever.
If the intent is to trigger an error to see how things look like in Dataflows when an error occurs, you could take a different path where you can create a Fabric item (like a Lakehouse), create a Dataflow that connects to a table, save and run the Dataflow, then rename the table in the lakehouse and try to run the Dataflow again. It should fail as you've changed the name of the table in the Fabric item after the Dataflow was correctly saved and no validations failed.
When you do the "save & run", if the save operation fails then you'll be able to check it through the "Check validation". If it fails, it means that the version that you tried to save didn't actually save, so you are using the previous version of your Dataflow.
You have a couple of ways now to check what your published Dataflow looks like. Here are a the list of ways to check them out from easiest to more complex:
- Opening the Dataflow: When you use Dataflow Gen2 with CI/CD capabilities, you can always discard any previously unsaved versions of your Dataflow and then open again the Dataflow which should match the version of the Dataflow that the pipeline can trigger for refresh
- Checking Git: As this is not an option for you, I'll just skip this one but its also one of the easiest ones where you could just check Git and see exactly what the M code looks like
- REST API: This is a bit more complex, but you can leverage the REST API endpoint for GET Dataflow definition. The actual M code for your is in the path: mashup.pq and you'll need to decode the payload which is in base64 to then see your full Dataflow mashup script.
In other words, if we find any issues with your Dataflow during the "save" operation, then it will simply not be committed whatsoever.
If the intent is to trigger an error to see how things look like in Dataflows when an error occurs, you could take a different path where you can create a Fabric item (like a Lakehouse), create a Dataflow that connects to a table, save and run the Dataflow, then rename the table in the lakehouse and try to run the Dataflow again. It should fail as you've changed the name of the table in the Fabric item after the Dataflow was correctly saved and no validations failed.
Ok, very clear !
I now fully understand how Dataflow Gen2 works, and it makes sense, especially compared to Gen1, which I was more familiar with before. That explains the differences I was seeing.
To confirm, I ran a test where I didn’t introduce an error in the Power Query itself (since that would prevent saving), but instead I introduced an error in the data source. That way, the Dataflow was successfully saved and then failed at runtime, exactly as expected. The error was properly raised and surfaced in my Data Pipeline.
Appreciate the clarification!