Skip Partitions during Processing

[rks]

Hi all,

I have a use case with some historical data that never updates. My idea was to throw all this into a partition of a history-dimension and tag this partition (maybe with certain external tools) as "don't process during table refresh". I am aware of the fact that I can create scripts or manually refresh certain partitions. But in this case it would be easier just to implement this information into the model.bim, rather than to create certain processing strategies.

Thank you all and best regards,

Konstantin

[Greg_Deckler]

@rks If you have a separate query for this then you can simply right-click the query and uncheck "Include in report refresh".

[rks]

Hi @Greg_Deckler , thanks for the answer. If I am not mistaked this is on the table level but not on the partition level. By default all partitions of a table are being processes. Is this flag also available on a partition level?

[Greg_Deckler · Accepted solution]

@rks I'm missing too much information to be very specific here. If you could provide more information I could be more specific. Let's say you have a single table with all of your historical and current data in it. Pick a point in time and create a query that pulls the historical information. Load. Disable refresh. Create a second query that pulls the current information. Setup incremental refresh on this query. Append the two queries. Hide the historical table in the data model. You could do something like that for example but, again, I don't even know what your source systems are or what you are truly trying to accomplish.

[rks]

Hi @Greg_Deckler  THank you very much for your detailed explanation. But all this is considering the processing strategy. My takeaway is that inside the model itself I can only disable refresh per table but not per partition.

My scenaio is this: 

* I have an Azure Analysis Service

* I have a dimension with around 1 million rows that can change on a regular basis

* I want to schedule a 30 minute refresh on the data set

* I have a legacy system (data source) that provides another 500.000 or so rows that will never change

* I wanted to save some load performance by neglecting the partition that hosts the historical data that will never update anymore

* I didn't want to update to job responsible for processing but at the same time wanted to gain a little processing-performance-boost by avoiding loading unneccassary data