Forum Discussion

Abhilash_P's avatar
Abhilash_P
Icon for Super User rankSuper User
2 years ago

IIncremental Refresh failure with minimal data load and full data load

Hi Everyone,
I have created below parameters in power query and trying to apply incremental refresh but not able execute incremental refresh.
IsMinimalLoad / IsMinimalLoadValue parameter is used to load minimul data into Power BI desktop.

DataSource for the report is Databricks,
Below are the applies steps
Native queries option is active till Rawdata step

 

7 Replies

  • is this a dataset or a dataflow?  You are seemingly missing the step where it filters for RangeStart/RangeEnd.

     

    What is your interpretation of "failure" ?

    • Abhilash_P's avatar
      Abhilash_P
      Icon for Super User rankSuper User

      Hi lbendlin,
      It's a dataset and tried applying filters (RangeStart and RangeEnd) but data is not being filtered as i have used select statement.

      • lbendlin's avatar
        lbendlin
        Icon for Super User rankSuper User

        That should work. Please show a sanitized version of your Power Query code. 

  • Hi lbendlin ,
    Below is the snapshot for RangeStart and RangeEnd

     

     

    Below are the Power query steps

    Below is the code in advance editor

     

    • lbendlin's avatar
      lbendlin
      Icon for Super User rankSuper User

      RangeStart and RangeEnd cannot both be inclusive.  One (usually RangeStart) needs to be inclusive, the other (usually RangeEnd) needs to be exclusive.  Otherwise you duplicate data across partitions.

      • Abhilash_P's avatar
        Abhilash_P
        Icon for Super User rankSuper User

        Hi lbendlin  ,
        Thanks for the reply
        I have changed RangeEnd to < insted of <=
        Still not able to perform incremental refresh

         


        I am using Azure databricks as my source and query folding is disabled

        Did you have any reference on implementing Incremental refresh with data bricks as the source..?
        Thanks a lot your response.