Forum Discussion

msprog's avatar
msprog
Advocate III
11 months ago
Solved

Copy activity dynamic mapping

Hi,

we have a data pipleine with  copy activity, we read data from SAP DSP and write to a lakehouse table. 

some of the SAP fields are TimeSPan which are not supported by lakehouse, so we enable explicity mapping and change the type to string. This works fine. 

 

Now we want to go a step further, and work towards creating a pipeline that dynamically constructs the json mapping - conforming to the source and sink tags that you see with the explicit mapping.

we read the source column types and if it timespan, we create ssink as physicalType: string. 

this dynamically consttructed json mapping is then passed to the copy activity. 

However, this doesn't work, the mapping doesn't get applied.  So wanted to check with the team here if they have done or faced some similar and if there is way to address this. 

 

thanks

 

  • Hello msprog ,

    I recommend you follow these steps (they helped me) :

     - Build a manual mapping.
     - Run the pipeline and check that it works.
     - Retrieve the mapping generated in the JSON code.
     - Delete the manual mapping and copy/paste the dynamically generated code into the mapping.


    Does it work now? 

    This allows us to rule out a problem with the automatic generation of JSON code.

    Have a nice day,

    Vivien

4 Replies

  • Hello msprog ,

    I recommend you follow these steps (they helped me) :

     - Build a manual mapping.
     - Run the pipeline and check that it works.
     - Retrieve the mapping generated in the JSON code.
     - Delete the manual mapping and copy/paste the dynamically generated code into the mapping.


    Does it work now? 

    This allows us to rule out a problem with the automatic generation of JSON code.

    Have a nice day,

    Vivien

  • v-menakakota's avatar
    v-menakakota
    Community Support

    Hi  msprog  ,
    Thanks for reaching out to the Microsoft fabric community forum. 

     

    I would also take a moment to thank vivien57   , for actively participating in the community forum and for the solutions you’ve been sharing in the community forum. Your contributions make a real difference. 
    I hope the below details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We’re always here to support you.

     

    Best Regards, 
    Community Support Team 

    • v-menakakota's avatar
      v-menakakota
      Community Support

      Hi msprog ,

      I hope the above details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We’re always here to support you.

       Best Regards, 
      Community Support Team  

      • v-menakakota's avatar
        v-menakakota
        Community Support

        Hi @msprog ,

        I hope the above details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We’re always here to support you.

         Best Regards, 
        Community Support Team