Forum Discussion

PanuO's avatar
PanuO
Helper II
2 years ago
Solved

Use iteration object in data copy mapping

I have dataflow which makes REST call to get different courses from API. For each course I need to get more detailed content. I have dataflow that uses Data Copy action to get all the courses and then I have For Each action that iterates every item received from API. I am able to do an another REST API call to get detailed course information, but the API response does not have the course identifier. I have it in the iteration item, but I cannot use it in mapping section. If I try to put item().id or {item().id} it produces an error:

ErrorCode=UserErrorSchemaMappingCannotInferSinkColumnType,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Data type of column 'id' can't be inferred from 1st row of data, please specify its data type in mappings of copy activity or structure of DataSet.,Source=Microsoft.DataTransfer.Common,'

 

Is there a way to use iteration object in mapping / should I do my dataflow in some another way? I have used the item().id in relative path, so I know it works there.

 

 

5 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi PanuO  - Thanks for using Fabric Community,

    As I understand you are trying to fetch data using API in Fabric Data Factory with the help of pipelines.

    Can you please confirm if your pipeline design is similar to below screen shot?


    Can you please share some details along with screenshots regarding your approach and issue.
    I would be able to help you better if you can provide these details.

    • PanuO's avatar
      PanuO
      Helper II

      That is quite close. I now have datalake as input, because it is faster to test that way (I have stored data in there so I don't want to fetch it all the time when I test this).

      The foreach has following Items setting: 

      @activity('Get courses from datalake').output.value
      This is the pipeline:

       

      • Anonymous's avatar
        Anonymous
        Not applicable

        Hi PanuO  - Thanks for your response.

        Can you please share a complete screenshot of for loop settings and copy data activity settings.

        Eg:
        For Loop -

         



        Copy Data -