Forum Discussion

manoj_0911's avatar
manoj_0911
Icon for Kudo Commander rankKudo Commander
2 months ago
Solved

Could a Derived DateTime Partition Column Affect Incremental Refresh Performance?

Hi Team, I noticed one key difference between our Missed Utterances report and our other reports that use Incremental Refresh. For most of our reports, the partition column comes directly from the ...
Need Help
  • Natarajan_M's avatar
    Natarajan_M
    2 months ago

    Hi manoj_0911 ,

    Can you check the amount of data you are processing in the refresh ?

    The issue could be because of the volume of data being processed during the initial refresh.

    A good first step would be to reduce the amount of data being loaded and test the refresh behavior.

    What I usually do is create a parameter called LoadData (True/False) and use it in the Power Query layer as shown below:

    Let 
     
    source 
    ....
     
    #"FirstNRows" = Table.FirstN(Source, 10),
        #"DataLoad" = if Loaddata then Source else #"FirstNRows"
     
    in 
    #"DataLoad"


    In Power BI Desktop, keep the parameter set to False by default to avoid encountering this issue during development.

    Publish the file with parameter = False and do the first refresh so the partitions are created . 

    Once the partitions are created, set the parameter to True in the Power BI Service and refresh each partition using SSMS.

     

     

    If this response was helpful, please consider giving it a kudo and marking it as the correct solution—it helps other community members find answers more quickly.

AnkitKukreja's avatar
AnkitKukreja
Icon for Super User rankSuper User
2 months ago

Hi! manoj_0911 

 

Generally the amount of data available in Dev and prod differs. Your deployment to prod could be pulling in billion rows whereas in dev it could be in thousands.
I would then recommend to maybe reduce the timeframe used for incremental refresh maybe several years to 18 months or so, just an example.