Forum Discussion
Incremental and Max Memory usage for a P1
Hi pallavi_r ,
Thank you for your answer, i configured into "large dataset" setting with no luck. It still say 7,2 GB from the admin center and 22 GB from the Fabric Metrics App. The one time full refresh work fine, it's the second one, in increment that fail (only for full dataset size, for half ot if no issue).
I have no calculated columns, the heaviest table is directly loaded from a azure synapse pool with no PowerQuery step except the filter on RangeStart/RangeEnd date.
I guess my last option is through XMLA endpoint but i don't know how to automatize the refresh process through SMSS or Tabular Editor.
Hi JohnnyDax
There won't be any difference in scheduling the refresh of dataset. it is irrespective of full refresh, incremental refresh, partition based refresh. XMLA end point Partitioned based resfresh is one time deployment activity. and after that we just need to trigger/schedule the refresh. it will do the incremental refresh.
You must have already done the desktop setting for incremental. What is the incremental range - is it weekly, monthly, quarterly.
1. Go to workspace setting. And copy the workspace connection and login to SSMS with this connection.
2.better to cleanse all the data by writing a view that we can modify outside of pbi to take one day range.
3.Click on full refresh from power bi service
4.It will create partitions as per your incremental configurations
5.Go to ssms->go to the dataset-> right click on partitions
6.individually run the partitions by clicking on the partion and select process data
7.After all partions are done, Right click on the entire dataset->process database->select process recalc
8.Now this is done, you can trigger or schedule the refresh is usual way. it would work incrementally
Few things to take note. I) Query folding should be taken care of. II) does not work well with git integration (this is what i had faced in the past, not sure if any new update is there).
I followed this video from guy in a cube - https://www.youtube.com/watch?v=5AWt6ijJG94
Please try this out and let me know if you have face any challenges.
Thanks,
Pallavi
Thanks,
Pallavi