Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
3 years ago

Dynamically calling REST APIs in Data Factory under Fabric

I am attempting to call dynamically an open API in Data Factory under Fabric. I connected to a REST API where I provided names and ids. I have the following pipeline:   My Lookup activity con...
Need Help
Pipelines
ajarora's avatar
ajarora
Microsoft Employee
3 years ago

Hello. Glad to know things worked.

For your follow up ask, if your ask is to use copy and write new data into lakehouse tables, we support "append" and "overwrite" mode already. But to handle duplicates you actually need UPSERT capability. This is a new functionality we are exploring, and if you are interested, can you please vote on this existing fabric idea about it: https://ideas.fabric.microsoft.com/ideas/idea/?ideaid=87f3d168-6022-ee11-a81c-6045bdc01ce4

 

If I misunderstood your ask, please clarify.

Anonymous's avatar
Anonymous
Not applicable
3 years ago

Basically, I have just looked at that 'append' and 'overwrite' concern Lakehouse Tables, but not Lakehouse Files (which is my case).

 

There are various options of Copy Behavior. Should I use them? Or any workarounds in terms of UPSERT logic?

  • ajarora's avatar
    ajarora
    Microsoft Employee
    3 years ago

    I dont see how you can use "Files" to be able to maintain a tabular data, ie rows with unique keys. Please ping me offline, I can guide you further on how to proceed on that. But hoping the original issue you reported is addressed.