Forum Discussion
Incremental Refresh
They can't both be inclusive. One has to be inclusive and the other has to be exclusive.
- stribor452 years agoPost Prodigy
what do you mean?
- lbendlin2 years agoSuper User
Your filter settings will cause data to bleed over across partitions. Needs to be like this:
RangeStart <= DateTime column < RangeEnd
- stribor452 years agoPost Prodigy
This is what I have which is saying filter where "requesteddate is bigger or equal to April 1 and requesteddate is smaller or equal to April 9. Sorry, can you maybe try to explain it differently because I do not understand.
- lbendlin2 years agoSuper User
This is what I have which is saying filter where "requesteddate is bigger or equal to April 1 and requesteddate is smaller or equal to April 9.That is wrong. Must be
filter where "requesteddate is bigger or equal to April 1 and requesteddate is smaller than
or equal toApril 9. - stribor452 years agoPost Prodigy
ok, that seems to work but I do have another question. If I have to adjust the report for example I want to change the formatting on the numbers going back to Dekstop to my model and add that to my visual I would need to save and republish. Would doing that mess up my partitions?
How would I know that partitions are successfully created?
My policy seems to be missing January 1 so to fix it I would need to change it to 100 days, save and refresh in Desktop and then publish and refresh again in Service?
- lbendlin2 years agoSuper User
Yes, any meta data change and report publish will reset the partitions, as discussed earlier. If you don't want that, use ALM toolkit to push the meta data changes without republishing
You can check the partition status with SSMS or DAX Studio or Powershell.
Change your archive policy to 4 months rather than 99 days
- stribor452 years agoPost Prodigy
I am in DAX studio (see the image). where is the partitioned data?
- lbendlin2 years agoSuper User
Advanced... View Metrics
or run the DMV query
select * from $SYSTEM.TMSCHEMA_PARTITIONS
- stribor452 years agoPost Prodigy
cool I was able to find that. when I schedule an automatic refresh even thought my query looks something like this only last 3 days will be actually pulled using API which is what i am using in my power query?
let Source = Json.Document(Web.Contents(url, [ RelativePath = "some/path", Headers=headers, Content = Text.ToBinary("{ ""filters"": [ { ""name"": ""requestedTime"", ""operator"": ""is"", ""value"": [ ""2024-01-01T00:00:00-04:00"", """ & currentDate & """ - lbendlin2 years agoSuper User
Where is the RangeStart and RangeEnd part of the query filter?
- stribor452 years agoPost Prodigy
only in this line as the last line of the power query code
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each [Column1.requestedTime] >= RangeStart and [Column1.requestedTime] < RangeEnd) - lbendlin2 years agoSuper User
that means you are loading all the data, all the time, and only at the very end filter the rows you want to put into the partition. This will result in all your data being pulled multiple times (once per each partition).
Unless this is a dataflow you want to put the range filter at the very beginning of your code, not the very end. Ideally your query should fold at that point (if it doesn't then performance is not as good as it could be).
- stribor452 years agoPost Prodigy
there is no other place to put it a few lines above is the data is coming in. See the code in the attached image. where would you place the filter?
- lbendlin2 years agoSuper User
In row 2
- stribor452 years agoPost Prodigy
in the image I have rows 37-50