Incremental Refresh Azure Blob

[vitoriamariabr]

Hi, I need help implementing incremental refresh on Azure Blob databases. I've already implemented it the standard way, but the incremental refresh takes forever and sometimes it doesn't even update, I don't think it's working correctly. I implemented it the basic way, creating the parameters and filtering RangeStart and RangeEnd in my date column (in azure blob the date column I used was Date modified) and I also tested it by inserting the incremental refresh filtering directly in the source:

"let
startDate = RangeStart,
endDate = RangeEnd,
Fonte =Table.SelectRows(AzureStorage.Blobs("https://xxxxxxxxx.blob.core.windows.net/xxxxx"),each [Date modified] >= startDate and [Date modified] <= endDate)".

However, the incremental update still doesn't work correctly. Can anyone help me?

olá! Preciso de ajuda para implementar atualização incremental em bases do Azure Blob. Já implementei da forma padrão, porém a atualização incremental demora uma eternidade e às vezes nem chega a atualizar, não acho que esteja funcionando corretamente. Cheguei a implementar da maneira básica, criando os parâmetros e filtrando RangeStart e RangeEnd na minha coluna de data (no azure blob a coluna de data que usei foi o Date modified) e testei também inserindo a filtragem da atualização incremental direto na fonte:

"let
startDate = RangeStart,
endDate = RangeEnd,
Fonte =Table.SelectRows(AzureStorage.Blobs("https://xxxxxxxxx.blob.core.windows.net/xxxxx"),each [Date modified] >= startDate and [Date modified] <= endDate)".

Porém, ainda assim a atualização incremental não funciona corretamente.

Alguém saberia me ajudar?

[lbendlin]

RangeStart and RangeEnd cannot be both inclusive.  Usually RangeStart is inclusive and RangeEnd is exclusive.

You cannot use Date columns, it must be DateTime or DateInteger.

[vitoriamariabr]

I see. My column is a DateTime. I'll adjust the RangeStart and RangeEnd, but I don't think that's the problem.

[lbendlin]

What is your reasoning for doing this in the first place?

[vitoriamariabr]

I have historical data in azure blob that is immutable, so I don't need to update it every time I need to update the report. That's why I want to implement incremental refresh, taking only the last 7 days, for example. But the big difficulty here is structuring the incremental refresh in the best way, and I'm not succeeding. I believe that the Azure Blob bases don't support query folding, and this may be considerably impacting the incremental update and I don't know what to do about it 😥😭

[lbendlin]

I think it is a design conflict. Azure Blobs are not supposed to be re-stored somewhere else.

Do you have a performance issue during regular refresh?