Copy activity dynamic mapping

[msprog]

Hi,

we have a data pipleine with  copy activity, we read data from SAP DSP and write to a lakehouse table. 

some of the SAP fields are TimeSPan which are not supported by lakehouse, so we enable explicity mapping and change the type to string. This works fine. 

Now we want to go a step further, and work towards creating a pipeline that dynamically constructs the json mapping - conforming to the source and sink tags that you see with the explicit mapping.

we read the source column types and if it timespan, we create ssink as physicalType: string. 

this dynamically consttructed json mapping is then passed to the copy activity. 

However, this doesn't work, the mapping doesn't get applied.  So wanted to check with the team here if they have done or faced some similar and if there is way to address this. 

thanks

[v-menakakota]

Hi  @msprog  ,
Thanks for reaching out to the Microsoft fabric community forum. 

I would also take a moment to thank @vivien57   , for actively participating in the community forum and for the solutions you’ve been sharing in the community forum. Your contributions make a real difference. 
I hope the below details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We’re always here to support you.

Best Regards, 
Community Support Team 

[v-menakakota]

Hi @msprog ,

I hope the above details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We’re always here to support you.

 Best Regards, 
Community Support Team  

[v-menakakota]

Hi @msprog ,

I hope the above details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We’re always here to support you.

 Best Regards, 
Community Support Team  

[vivien57 · Accepted solution]

Hello @msprog ,

I recommend you follow these steps (they helped me) :

 - Build a manual mapping.
 - Run the pipeline and check that it works.
 - Retrieve the mapping generated in the JSON code.
 - Delete the manual mapping and copy/paste the dynamically generated code into the mapping.

Does it work now? 

This allows us to rule out a problem with the automatic generation of JSON code.

Have a nice day,

Vivien