Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
4 years ago

Incremental Refresh for Delta Tables / Databricks

Hi guys,

 

I have quite big tables stored as Delta Tables in Azure Storage Account and also as Databricks Tables, that I need to connect to (Import mode!).

As those tables are quite big and the daily refresh must not take long, incremental refresh is definetely needed. Unfortunately it seems like query folding is not provided for delta tables or Databricks.

Do you have an idea how I can implement incremental refresh for those tables? Or maybe also an alternative idea for the storage of the tables is welcome.

 

Best,
Massimow