Forum Discussion
Fabric copy activity fails when JSON source is missing attributes
Hi Ashutosh_A,
Thank you for reaching out to the Microsoft fabric community forum.
It looks like the Copy activity is expecting all the mapped fields to be present in the API response. If any struct is missing in a particular response, the activity fails because the defined mapping does not find that field in the JSON.
Copy activity works well when the schema is consistent, but it can fail when nested fields are optional or missing in some responses. In this case, using a Notebook or Dataflow Gen2 may help, since you can handle the JSON more flexibly and set NULL or default values when fields are not present.
This way the pipeline will not fail even if some attributes are missing in the API response. Let us know if you need any more details.
Kindly refer to the below documentation links for better understanding:
Configure REST in a copy activity - Microsoft Fabric | Microsoft Learn
Hope the above provided information help you resolve the issue, if you have any further concerns or queries, please feel free to reach out to us.
Regards,
Community Support Team.
Hi Ashutosh_A,
The best way to do this would be to use a notebook instead of a copy activity, then you can write code to handle missing fields however you like.
If you need a low/no code solution, you might have more luck with a dataflow gen 2.