Forum Discussion
Advocate IAutomating Delta Table Updates from folders in Lakehouse
Hi,
I suppose the shortcuts you create from the ADLS storage are landed in the Files section of your Lakehouse. Those files are always up-to-date with the source (ADLS storage). To get an updated Delta table, you should use one of the options to copy data from the Files section to the Tables. This can be done with copy data in a data pipeline, DataFlow Gen2 or a notebook where you write your own (Python) script.
Additionally, you could use Data Activator to automatically run the pipeline or notebook once a new file in ADLS is landed. Trigger Fabric items - Microsoft Fabric | Microsoft Learn, but it depends how your data is changed in the ADLS if the trigger exists.
Solution SageHi,
I suppose the shortcuts you create from the ADLS storage are landed in the Files section of your Lakehouse. Those files are always up-to-date with the source (ADLS storage). To get an updated Delta table, you should use one of the options to copy data from the Files section to the Tables. This can be done with copy data in a data pipeline, DataFlow Gen2 or a notebook where you write your own (Python) script.
Additionally, you could use Data Activator to automatically run the pipeline or notebook once a new file in ADLS is landed. Trigger Fabric items - Microsoft Fabric | Microsoft Learn, but it depends how your data is changed in the ADLS if the trigger exists.