Forum Discussion
Fabric copy activity fails when JSON source is missing attributes
Hi Ashutosh_A,
Thank you for reaching out to the Microsoft fabric community forum.
It looks like the Copy activity is expecting all the mapped fields to be present in the API response. If any struct is missing in a particular response, the activity fails because the defined mapping does not find that field in the JSON.
Copy activity works well when the schema is consistent, but it can fail when nested fields are optional or missing in some responses. In this case, using a Notebook or Dataflow Gen2 may help, since you can handle the JSON more flexibly and set NULL or default values when fields are not present.
This way the pipeline will not fail even if some attributes are missing in the API response. Let us know if you need any more details.
Kindly refer to the below documentation links for better understanding:
Configure REST in a copy activity - Microsoft Fabric | Microsoft Learn
Hope the above provided information help you resolve the issue, if you have any further concerns or queries, please feel free to reach out to us.
Regards,
Community Support Team.
Hi Ashutosh_A,
Thank you for reaching out to the Microsoft fabric community forum.
It looks like the Copy activity is expecting all the mapped fields to be present in the API response. If any struct is missing in a particular response, the activity fails because the defined mapping does not find that field in the JSON.
Copy activity works well when the schema is consistent, but it can fail when nested fields are optional or missing in some responses. In this case, using a Notebook or Dataflow Gen2 may help, since you can handle the JSON more flexibly and set NULL or default values when fields are not present.
This way the pipeline will not fail even if some attributes are missing in the API response. Let us know if you need any more details.
Kindly refer to the below documentation links for better understanding:
Configure REST in a copy activity - Microsoft Fabric | Microsoft Learn
Hope the above provided information help you resolve the issue, if you have any further concerns or queries, please feel free to reach out to us.
Regards,
Community Support Team.
Hi Ashutosh_A,
I hope the information provided above assists you in resolving the issue. If you have any additional questions or concerns, please do not hesitate to contact us. We are here to support you and will be happy to help with any further assistance you may need.
Regards,
Community Support Team.
Hi Ashutosh_A,
I hope the information provided above assists you in resolving the issue. If you have any additional questions or concerns, please do not hesitate to contact us. We are here to support you and will be happy to help with any further assistance you may need.
Regards,
Community Support Team.