Forum Discussion

P3Sven's avatar
P3Sven
Advocate I
4 years ago

Dataverse | Incremental Refresh vs. Dataverse 80mb limit

Hi lovely community,

 

I have a dataverse with some fact tables including about 1 million records and 100 columns.

So its not possible to load all records with one request per Dataverse Connector cause of 80mb limit.

 

So I just wondering how does it works with incremental refresh in Power BI service?

Like one time 9 years and daily scheduled 1 year?

 

Does someone have some experience with it?

 

BR,

Sven

Need Help

2 Replies

  • v-yalanwu-msft's avatar
    v-yalanwu-msft
    Community Support
    4 years ago

    Hi, P3Sven ;

    1) From what I understand, incremental refresh is set up on the desktop and is mainly used during service refresh to improve refresh performance.
    2) Can you tell me whether your report is published to service? Through official documentation, This error can occur with the Dataverse connector when you run or design a report with a result set greater than 80 MB. TDS has a result set size limit of 80 MB.

     


    To work around this limit, optimize the query adding filters and dropping columns so that the query returns less data.
    3) To sum up, I think your data may be too large to load, however incremental refresh can be loaded, just to improve the refresh performance on the service, so you can  filters and dropping columns so that the query returns less data as stated on the official document. 

    4)In additional, here is some link about incremental refresh that you can refer.

    https://docs.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-configure

    Best Regards,
    Community Support Team_ Yalan Wu
    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

    • P3Sven's avatar
      P3Sven
      Advocate I
      4 years ago

      Hi v-yalanwu-msft ,

       

      sorry for late reply, had some trouble days.

      Thank you for your information!

       

      My Case:

      I've many Dataverse Table I need to query with Power BI with long time range. With Power BI Desktop I've built a function to make Dataverse Request per Year and after merging it to one Table (so its just a workarround for the 80mb limit.)

       

      I've tested setting Incremental Refresh for one of these Dataverse table (incl. RangeStart and RangeEnd Parameter, 9 years for one time query and 1 year for daily query).

      So in Power BI Desktop I just query one month with it (cause of the Parameters), of course it works fine.  

      After publishing I've recognized I had to refresh dataset one time manually and it worked without Errors like the 80mb Limit.

       

      Ive checked the whole table per editing the report online, all columns incl. 10years are available..

       

      So I guess 80mb Limit for Dataverse is not for the Power BI Service? Is that true? 

       

      BR,

      Sven