Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Grow your Fabric skills and prepare for the DP-600 certification exam by completing the latest Microsoft Fabric challenge.

Reply
Winnie2024
New Member

Is Fabric pipeline copy activity dynamic mapping support copying only partial source data to sink?

I have some template files, each with different column names. And users may need to customize new files with exsting tempaltes by combination randomly. Here is my design:

 

I create a delta table containing all the columns of all template fiels and define all the columns mapping  in one file like below:

 

{
"type": "TabularTranslator",
"mappings": [
{
"source": {
"name": "T1_F1",
"type": "String",
"physicalType": "String"
},
"sink": {
"name": "A_1",
"physicalType": "string"
}
},
{
"source": {
"name": "T1_F2",
"type": "String",
"physicalType": "String"
},
"sink": {
"name": "A_2",
"physicalType": "string"
}
},
{
"source": {
"name": "T2_F1",
"type": "String",
"physicalType": "String"
},
"sink": {
"name": "B_1",
"physicalType": "string"
}
},
{
"source": {
"name": "T2_F2",
"type": "String",
"physicalType": "String"
},
"sink": {
"name": "B_2",
"physicalType": "string"
}
}

],
"typeConversion": true,
"typeConversionSettings": {
"allowDataTruncation": true,
"treatBooleanAsNumber": false
}
}

 

and put it in the paramtere for the Fabric pipeline with parameter key"mapping" as below

Screenshot 2024-04-12 at 17.57.55.png

 

And define the dynamic mapping as below:

Screenshot 2024-04-12 at 17.57.45.png

 

 

I can copy the files with ALL columns defined in the prediffined-mapping. But failed with any file with only partial columns. 

Take the example above, I can copy copy files with columns "T1_F1" ,"T1_F2","T2_F1" and "T2_F2".

But I CAN NOT copy file with only "T1_F1"  and "T1_F2"

How to do solve this?

4 REPLIES 4
v-nikhilan-msft
Community Support
Community Support

Hi @Winnie2024 
Thanks for using Fabric Community.
At this time, we are reaching out to the internal team to get some help on this. We will update you once we hear back from them.
Thanks 

Hi @Winnie2024 
Can you please confirm what is the error you are getting? A screenshot would help.
Thanks

Hi @Winnie2024 
We haven’t heard from you on the last response and was just checking back to see if you can provide the details asked above.

Thanks

Hi @Winnie2024 
We haven’t heard from you on the last response and was just checking back to see if you can provide the details asked above.

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

MayFBCUpdateCarousel

Fabric Monthly Update - May 2024

Check out the May 2024 Fabric update to learn about new features.