Forum Discussion

ClaireCC's avatar
ClaireCC
Frequent Visitor
5 years ago

First incremental refresh large historical data

Hi Community,

 

I have a questions on the first incremental refresh launched in Power Bi Service.

 

My dataset contains 3 tables (sales, stock, stock movments) with incremental parameters RangeStart end RangeEnd defined.

I need to load 2 years of historical data and refresh 10 days.

 

All works perfectly (I have done some tests with smaller data set) but every time I launch the first incremental refresh, I have a timeout issue because it exceeds the 2 hours time limit in Pro Licence.

 

Is there a way to avoid this limit for the first incremental refresh that need to import all the historical data ?

 

I have seen some solutions with XMLA endpoint but it's for Premium licences...

 

Any help would be very appreciated !

 

Thanks in advance

Need Help
Tips and Tricks

3 Replies

  • arvindsingh802's avatar
    arvindsingh802
    Community Champion
    5 years ago

    You can break the data load using dataflow 1 table in each dataflow, and pull data to your dataset

    Make sure that datalfows first refresh at different time, it will reduce load at your source and gateway and speed up the things.

     

    If this post helps, then please consider Accept it as the solution and give it a thumbs up.

  • ClaireCC's avatar
    ClaireCC
    Frequent Visitor
    5 years ago

    Dataflows incremental refresh is only available for Premium licence or I need to implement incremental refresh.
    We integrate stocks and sales every day, several times a day. I can't do a full import every time it's too long. Even with 1 table per dataflow, my stock table is too big to do a full import every time. 

    • Anonymous's avatar
      Anonymous
      Not applicable
      3 years ago

      Hey ClaireCC 
      is your problem solved now?
      if yes then can you please suggest me because im also suffering from a similar problem

      Thanks and Regards
      Mihir