Forum Discussion
Copy data - Mapping section is Collection reference option
Hi,
Trying to load data from a rest API using "Copy data". Got it working with pagination and everything, but can't access the real data as the "Collection reference" is missing in the gui...?
Data coming through the API looks like this. Would like to get the data inside `values`.
[
{
"fullResultSize": 1383,
"from": 0,
"count": 1000,
"versionDigest": "'If-None-Match' header not specified",
"values": [
{
"id": 100686570,
"version": 6,
"url": "example.com",
"project": {
"id": 123456,
"url": "example.com/133833984"
},
}
]
}
]
Clicking "view source" on the object it is there but empty
{
"collectionReference": "",
"typeConversion": true,
"typeConversionSettings": {
"allowDataTruncation": false,
"treatBooleanAsNumber": false
},
"columnFlattenSettings": {
"treatArrayAsString": false,
"treatStructAsString": false,
"flattenColumnDelimiter": "."
}
Any suggestions? Thanks!
13 Replies
We're missing this option too.
We use this option a lot in pipelines in Azure Synapse. Currently we can't use Fabric because of the absence of this option.
Any updates on this? Or is there an Idea to vote for?
UPDATE
I found out that if you choose an 'External' sink first, the option will appear. You can set it at that time and change the sink afterwards to 'Workspace'. The setting 'Collection reference' is still set in the JSON definition.
Then just map like you normally would.
Hi ribba
Thanks for using Microsoft Fabric Community.
At this time, we are reaching out to the internal team to get some help on this.
We will update you once we hear back from them.Appreciate your patience.
Thanks
Thanks !
Hi! It's been a while since this thread was updated, any news on supporting "Collection reference" in Fabric?
So when just the source is selected the collection reference appears
But as soon as I select a destination, it disappears, regardless of the destination:
Found a workaround. Roaming around with the json for the pipeline i found that by changing the value of typeConversion from true to false enables the collation reference. Assuming this will be fixed in the future this pipeline may break, but it works for now.
v-cboorla-msft Any update on this?
Hi ribba
Apologies for the issue that you are facing here.
This might require a deeper investigation from our engineering team about your workspace and the logic behind it to properly understand what might be happening.
Please go ahead and raise a support ticket to reach our support team:
https://support.fabric.microsoft.com/support
Please provide the ticket number here as we can keep an eye on it.
Thanks.
Any update on this? I'm finding a similar problem with the mapping. I think it's in part to the copy data activity not being able to recognize the array data type. I have never used Azure Data Factory but I see that Collection Reference and Advanced Editor exist in ADF but not in Microsoft Fabric. Is there a work around for this?
I'm also facing similar issue 'collectionReference' property is not availble for copy data activity in Microsoft Fabric. Any plans to enable this property near future ?
Hi,
This is stopping me as well. Any ideas on how to work around it?
I haven't seen a post on this, in a while. Did this somehow get solved?
