Selective update of lakehouse table in Fabric

[ebjim]

I use a dataflow gen2 to check for changes between a lakehouse table and an onprem table. I can identify and isolate the rows that have been changed but I am having difficulty determining if there is a way in Fabric to selectively update the lakehouse table. The dataflow doesn't offer such an option. I have read articles about replaceWhere in Databricks but it does not look like Fabric can handle it. In any case, is there a way to perform selective updates of a lakehouse table based on records of another lakehouse table?

[v-nikhilan-msft · Accepted solution]

Hi @ebjim 
Thanks for using Fabric Community.
I think we can use Incremental load in your case. Incrementally amassing data in a data destination requires a technique to load only new or updated data into your data destination. This technique can be done by using a query to filter the data based on the data destination.
Please refer to this document for more information: Pattern to incrementally amass data with Dataflow Gen2 - Microsoft Fabric | Microsoft Learn

Hope this helps. Please let me know if you have any further queries.

[v-nikhilan-msft]

Hi @ebjim 
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. Otherwise, will respond back with the more details and we will try to help.
Thanks