Forum Discussion
Dynamically calling REST APIs in Data Factory under Fabric
ajarora Thanks! I have just suceeded. Missing brackets were the case.
My current implementation involves obtaining data from multiple APIs simultaneously every day. I want to implement a process that does not rewrite or overwrite new data but appends it, taking into consideration the possibility of duplicates in consecutive API calls. Is it possible to achieve this using Data Factory in Fabric?
Hello. Glad to know things worked.
For your follow up ask, if your ask is to use copy and write new data into lakehouse tables, we support "append" and "overwrite" mode already. But to handle duplicates you actually need UPSERT capability. This is a new functionality we are exploring, and if you are interested, can you please vote on this existing fabric idea about it: https://ideas.fabric.microsoft.com/ideas/idea/?ideaid=87f3d168-6022-ee11-a81c-6045bdc01ce4
If I misunderstood your ask, please clarify.
Basically, I have just looked at that 'append' and 'overwrite' concern Lakehouse Tables, but not Lakehouse Files (which is my case).
There are various options of Copy Behavior. Should I use them? Or any workarounds in terms of UPSERT logic?
I dont see how you can use "Files" to be able to maintain a tabular data, ie rows with unique keys. Please ping me offline, I can guide you further on how to proceed on that. But hoping the original issue you reported is addressed.