Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
ribba
Regular Visitor

Copy data - Mapping section is Collection reference option

Hi,

 

Trying to load data from a rest API using "Copy data". Got it working with pagination and everything, but can't access the real data as the "Collection reference" is missing in the gui...?

 

ribba_0-1702338905533.png

 

Data coming through the API looks like this. Would like to get the data inside `values`.

[
  {
    "fullResultSize": 1383,
    "from": 0,
    "count": 1000,
    "versionDigest": "'If-None-Match' header not specified",
    "values": [
      {
        "id": 100686570,
        "version": 6,
        "url": "example.com",
        "project": {
          "id": 123456,
          "url": "example.com/133833984"
        },
      }
    ]
  }
]

 

Clicking "view source" on the object it is there but empty

{
"collectionReference": "",
"typeConversion": true,
"typeConversionSettings": {
"allowDataTruncation": false,
"treatBooleanAsNumber": false
},
"columnFlattenSettings": {
"treatArrayAsString": false,
"treatStructAsString": false,
"flattenColumnDelimiter": "."
}

 

Any suggestions? Thanks!

 

9 REPLIES 9
nbjdeunk
Frequent Visitor

We're missing this option too. 

 

We use this option a lot in pipelines in Azure Synapse. Currently we can't use Fabric because of the absence of this option.

 

Any updates on this? Or is there an Idea to vote for?

 

UPDATE

I found out that if you choose an 'External' sink first, the option will appear. You can set it at that time and change the sink afterwards to 'Workspace'. The setting 'Collection reference' is still set in the JSON definition. 

Then just map like you normally would. 

emcus
Frequent Visitor

Hi,

This is stopping me as well. Any ideas on how to work around it?

PrakashTamil
New Member

I'm also facing similar issue 'collectionReference' property is not availble for copy data activity in Microsoft Fabric. Any plans to enable this property near future ?

noahadmcdonald
Frequent Visitor

Any update on this? I'm finding a similar problem with the mapping. I think it's in part to the copy data activity not being able to recognize the array data type. I have never used Azure Data Factory but I see that Collection Reference and Advanced Editor exist in ADF but not in Microsoft Fabric. Is there a work around for this?

ribba
Regular Visitor

@v-cboorla-msft Any update on this?

Hi @ribba 

 

Apologies for the issue that you are facing here.

This might require a deeper investigation from our engineering team about your workspace and the logic behind it to properly understand what might be happening. 

Please go ahead and raise a support ticket to reach our support team:

https://support.fabric.microsoft.com/support
Please provide the ticket number here as we can keep an eye on it.


Thanks.

v-cboorla-msft
Community Support
Community Support

Hi @ribba 

 

Thanks for using Microsoft Fabric Community.
At this time, we are reaching out to the internal team to get some help on this.
We will update you once we hear back from them.

Appreciate your patience.

 

Thanks

Hi! It's been a while since this thread was updated, any news on supporting "Collection reference" in Fabric?

Thanks !

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

March 2024 FBC Gallery Image

Fabric Monthly Update - March 2024

Check out the March 2024 Fabric update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.