Mashup engine error when refreshing Dataflow Gen 2 with Lakehouse as destination

[MG86]

I am struggling with the following, perhaps it's just a bug in a product that's in beta, but hopefully someone can help:

I have created a Gen2 dataflow in which I get data from a REST/JSON Web API. The data loads correctly in the online power query, I can do all transformations etc. I've built three tables with this API that have their corresponsing table as destination in a Lakehouse. The first two tables give no error, but the third keeps returning the following error:

Mashup Exception Expression Error Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Expression.Error: We can't convert values of this type to JSON. Details: Reason = Expression.Error

I tried tons of things to solve this. But despite the query being build up the same as the other tables, despite it just being a few fields of which I;ve validated the data types and despite the 1:1 column mapping to lakehouse it keeps returning the same error. I even tried to put everything as a text column and still. What's especially weird is that it does not occur in Power BI Desktop and also not when previewing all data in PQ. It only happens on publishing. Even weirder is that 1 out of 10 times it does succeed,

Does the mashup engine perhaps struggle with the steps inbetween whereas PQ does not? Could it be caused by navigating through nested objects in the JSON response? 

The PQ Components are simply:

- Source = Json.Document(Web.Contents())),
- Navigation = Source[Folder]
- #Converted to table = Table.FromList()
- #Expanded Column1 = Table.ExpandRecordColumn()

Nothing fancy, and it loads everywhere except in this Dataflow refresh.

Any ideas?

[gthill]

I am struggling with a similar issue - MashupException.Error: Expression.Error: We can't convert values of this type to JSON. Details: Reason = Expression.Error

The Dataflow appears to be working fine in the editor, but when it is published and refreshed it fails with this error.

The Dataflow extracts data from Excel files via an ADLS Gen2 shortcut in the Lakehouse.

I've tried various data types, removing any spaces and special characters from column names, removing columns etc. but I'm out of ideas now and haven't been able to find a way to narrow this error down further - more details in the erorr message or a method to debug would be much welcomed.

Any suggested solutions please?

[gthill]

OK - so I created a new Dataflow and copied the M queries across from the one which is failing and this new one works fine.

I assume there is a bug or something corrupting the original Dataflow. 

Not an ideal solution, but might help others as a workaround until the root cause can be identified.

[lbendlin]

You can raise an issue at https://community.fabric.microsoft.com/t5/Issues/idb-p/Issues . If you have a Pro license you can consider raising a Pro ticket at https://admin.powerplatform.microsoft.com/newsupportticket/powerbi