Forum Discussion
Update backfill in chunks for incremental refresh
I'm using a large table as a dataset in Power BI. I applied an incremental refresh to this dataset, but after publishing with the initial scope defined by the RangeStart and RangeEnd, it requires a one-time back fill of archived data. I am running into connection and timeout issues with this back fill. Is there a way to run this back fill in smaller chunks?
Use CSV or Parquet files as fake partitions and then append them in Power Query. Yes, you will reload all the data each time but both CSV and Parquet are ingesting very fast.
5 Replies
Use the XMLA endpoint with tools like SSMS and then refresh individual partitions selectively.
This is a premium per user workspace but we do not have premium capacity. From what I've read, XMLA requires premium capacity, is this true? If so, is there a way to partition the data load outside of premium capacity?
Use CSV or Parquet files as fake partitions and then append them in Power Query. Yes, you will reload all the data each time but both CSV and Parquet are ingesting very fast.
