Forum Discussion
Custom partitioning with Tabular editor
There's that pesky problem of meta data changes. Whenever you change the data model, or even the report design, and publish to the workspace without further thought, a couple of things will happen.
- The incremental refresh partitions will be replaced with the single partition you just uploaded
- a full refresh will be initiated, across all currently specified partitions (depending on your granularity)
- after that is completed successfully your regular incremental refresh will resume.
To avoid that you need to choreograph meta data changes extremely carefully, push them with ALM Toolkit WITHOUT refresh, and then manually refresh the affected partitions.
Thank you so much.
Yes, we are reducing the size of the database to improve performance.
The error message says: "This operation was canceled because there wasn't enough memory to finish running it."
By connecting PBI dataset to the 30-month DB, and then partitioning it in tabular editor, followed by refreshing each partition separately using PowerShell (XMLA endpoint), in the first try, since the partitions are empty, it took around 4 minutes to refresh each partition, but subsequent refreshes took less than 20 seconds (all using automatic refresh type). This led me to think that using the automatic refresh type would be similar to employing "detect data change".
I always get that "not enough memory" error in PBI service, and I'm trying to divide all tables in multiple PowerShell batches and partitioning the large tables and refreshing partitions sequentially to avoid that error.
Thanks again for the comment.
There's that pesky problem of meta data changes. Whenever you change the data model, or even the report design, and publish to the workspace without further thought, a couple of things will happen.
- The incremental refresh partitions will be replaced with the single partition you just uploaded
- a full refresh will be initiated, across all currently specified partitions (depending on your granularity)
- after that is completed successfully your regular incremental refresh will resume.
To avoid that you need to choreograph meta data changes extremely carefully, push them with ALM Toolkit WITHOUT refresh, and then manually refresh the affected partitions.