Forum Discussion

KNO's avatar
KNO
Helper I
4 years ago

Delta Lake and incremental refresh - how?

HI

I have a Delta Lake dataset called 'hourdata'. It contains 150 million rows and consumes alot of memory. I have tried to do incremental refresh configuration in Power BI Desktop top with no success. Delta Lake does not seem be support Query Folding. So I have only one partions  at the moment. 

My question is as follow

  • How can I break down the dataset in smaller partition, without changing data source. 

 

Thanks 

 

Need Help

1 Reply

  • lbendlin's avatar
    lbendlin
    Super User
    4 years ago

    "Delta Lake does not seem to support Query Folding. So I have only one partition at the moment. "

     

    These things are not related.  You don't need Query Folding for incremental refresh (although it would help).  What you do need is a step in your Power Query that includes a RangeStart and RangeEnd filter.  Do you have that?