Forum Discussion
Fabric Dataflow Gen2 – Refresh succeeds despite Power Query error
- 1 year ago
When you do the "save & run", if the save operation fails then you'll be able to check it through the "Check validation". If it fails, it means that the version that you tried to save didn't actually save, so you are using the previous version of your Dataflow.
You have a couple of ways now to check what your published Dataflow looks like. Here are a the list of ways to check them out from easiest to more complex:
- Opening the Dataflow: When you use Dataflow Gen2 with CI/CD capabilities, you can always discard any previously unsaved versions of your Dataflow and then open again the Dataflow which should match the version of the Dataflow that the pipeline can trigger for refresh
- Checking Git: As this is not an option for you, I'll just skip this one but its also one of the easiest ones where you could just check Git and see exactly what the M code looks like
- REST API: This is a bit more complex, but you can leverage the REST API endpoint for GET Dataflow definition. The actual M code for your is in the path: mashup.pq and you'll need to decode the payload which is in base64 to then see your full Dataflow mashup script.
In other words, if we find any issues with your Dataflow during the "save" operation, then it will simply not be committed whatsoever.
If the intent is to trigger an error to see how things look like in Dataflows when an error occurs, you could take a different path where you can create a Fabric item (like a Lakehouse), create a Dataflow that connects to a table, save and run the Dataflow, then rename the table in the lakehouse and try to run the Dataflow again. It should fail as you've changed the name of the table in the Fabric item after the Dataflow was correctly saved and no validations failed.
Could you check on Git your pq file and see if it contains the step that introduces the error?
in principle, the save validation should prevent you from saving your dataflow hence that version of the dataflow shouldn't be used for refresh purposes. Therefore, what you're running in a pipeline is not actually the dataflow with the error that you introduced but probably a prior version of it.
do please let us know what you see on git