Forum Discussion
DataFlow Refresh Error and Fix for added Fields in Dynamics 365
I had the same problem but figured out the root cause....its the model.JSON file that gets updated with the schema of your dataflow. If you have a new column that gets added to your dataflow from the source (whether its another unlinked dataflow or a select * statement from a database), dataflows doesnt seems to auto-support schema drift.
High level process of what's happening
- Data source gets changed to now have 11 columns
- You refresh dataflow without editing to validation and save which triggers the model.JSON file to update
- Query dataflow from PBI Desktop: model.JSON file says to expect 10 columns but the actual CSV data that JSON file points to has 11 columns... thus you get this error.
If you are using disable load to consume data from other dataflows, using premium linked entities would likely fix the problem since metadata is linked instead of duplicated. But in your scenario it sounds like in Dynamics the scema drifted and Dataflows doesn't yet support source schema drift since the model.JSON is only compiled at validation of your dataflow when you save.
Workaround IF the source of your dataflow changes (i.e. more/less columns)
- edit dataflow (no actual edits required)
- click Validate and Save
- Refresh your dataflow after saving
- Refresh Power BI report and your new column will be available without error
You can validate all of this happens by reviewing the exported JSON file before valdiating and saving your dataflow and then viewing again after saving...you'll noticed that before the JSON did not having the new column and after, it will.
I made a copy of an existing report, removed one column, and got this error when I refreshed on PBI desktop.
Where do I go to edit dataflow ?