Forum Discussion
Solved
Error ingesting json files in warehouse table
Hi All, We have a pipeline that uses an API to get data from one of our suppliers. It will create a number of json files, which we then ingest into a warehouse table so we can ETL, join, upsert ...
Good morning All,
A small update, which is not really and update, but MS came back indicating they need more time and that the issue encountered seems to be more complex than expected.
v-kpoloju-msft We are already writing the data to a raw json file, this allowed us to identify the issue quick.
I will be looking into python as a notebook, as soon as i get a chance.
It is the copy activity to the lakehouse that is actaully throwing the error, preventing us from loading like we used to (this pipeline has worked fine for a least a year). The only thing we were able to do was change the 1st copy step to flatten the whole json array into one single column and next use a notebook.
Cheers
Hans