Forum Discussion
How implementing an incremental data loading in Data Factory
Hi, in Data Factory of Microsoft Fabric how I could implement an incremental data load? In Azure Data Factory I could use a mapping data flow to accomplish a such task. In Fabric is there the good...
Depending on your requirement of source and data structure, availability you can try to see if it can be achieved within pipeline using UI, however in scenarios requiring larger control and complex mechanism you can try to use dataflow. You can also use pipeline with spark code of your choose provide much more control in CICD. It all would come down to the mechanisms in place, existing expertise available and the path organization would like to adapt.
Hi, for example to ingest a customer table to handle as a SCD2 reading data from an on-premise source (e.g. Oracle, csv files).
Thanks