Forum Discussion
Error ingesting json files in warehouse table
A final update for now at least: Microsoft has confirmed this is a bug, they hope to fix in the 1st quarter of 2026.
In the mean time they suggest to ingest it as a string and than use a notebook to parse the json array back into its original columns, or change the source, which for us wasn't an option.
Since it took ages for them to got to this stage I had already figured out I could use a notebook to read the json files in the lakehouse, read them into a dataframe, and when all files were read, write that dataframe to a table, in my case with an option "overwrite" since this is just a raw staging table.Final kicker: They want me to give permission to archive the case (no doubt because of their KPI's), but the bug is not registered on MS's known issue list. I want them to document it there, but they for some reason don't want to do that, they want to keep it internal, so now my Dutch stubberness is playing up .
The python code was quite simple in the end, even though we had multiple nested arrays. I used co-pilot to do the heavy lifting for me since i had to alias about 30 columns since i had to put the data in an existing table and the names needed to match
These are the libraries I needed:
Folder path to read json files
Exploding the main body of the json (just part of it, as example)
Thats it in a nutshell
Hi smeetsh,
Thank you for reaching out to the Microsoft Fabric Community Forum and providing the details and the JSON examples.
The error you are seeing is happening because the testResults field is sometimes an array (with values) and sometimes null. The pipeline expects it to always be an array, so when it encounters null, it fails with the “Not able to get enumerator for non-array” error. This wasn’t enforced as strictly before, which is why it may have only started breaking recently.
To fix this, the recommended approach is to normalize the JSON before ingestion so that null values are replaced with empty arrays []. This keeps the schema consistent and prevents the pipeline from failing. https://learn.microsoft.com/en-in/azure/data-factory/copy-activity-schema-and-type-mapping
As a quick workaround, you can also enable Skip incompatible rows under Copy Activity → Settings → Fault tolerance but do note that this will drop any rows where testResults is null.
https://learn.microsoft.com/en-in/azure/data-factory/copy-activity-fault-tolerance
For a long-term stable solution, I would suggest going with the normalization option, so you don’t risk losing records when the source sends null.
Hope this clears it up. Let us know if you have any doubts regarding this. We will be happy to help.
Thank you for using the Microsoft Fabric Community Forum.
Hi smeetsh,
Just checking in to see if the issue has been resolved on your end. If the earlier suggestions helped, that’s great to hear! And if you’re still facing challenges, feel free to share more details happy to assist further.
Thank you.
I have tried the above solution, but it doesn't work.
I have a case open with MS, according to co-pilot the way Fabric handles json has become more strict since they launched the preview of the new activity copy job. But I can't find any real resource to confirm this. I will be pushing MS for an answer on that.Right now I am trying the solution tayloramy suggested, which puts the json as a string in a column and than parsing it. That is going to be a bit of a learning curve for me though. I will keep this thread updated if I find a solution and how i have done it.
Hi smeetsh,
Thanks for getting back with an update. It sounds like you are already exploring a solid workaround with tayloramy, suggestion of storing the JSON as a string and then parsing it totally understandable that it might take a little learning curve.
Good call on keeping the case open with Microsoft as well, since their confirmation on the stricter JSON handling will be the most reliable path forward. Please do share back here once you hear more or have success with your approach, I’m sure it will help others running into the same issue.Thank you.
Hi smeetsh,
Just checking in to see if the issue has been resolved on your end. If the earlier suggestions helped, that’s great to hear! And if you’re still facing challenges, feel free to share more details happy to assist further.Thank you.
Hi smeetsh,
Hope you had a chance to try out the solution shared earlier. Let us know if anything needs further clarification or if there's an update from your side always here to help.Thank you.