Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
lerecie
Frequent Visitor

Expand Dates in Large Data Set - PowerQuery and Dax

I have a large data set in a proper star schema. However, i'm struggling with some reports out of it.

I'll explain what i'm currently doing, and hopefully someone can tell me a better way (or more elegant) way to accomplish something similar.

 

Data set:

Screenshot 2021-03-29 114022.png

 

There's many more tables that have proper relationships (Aprox 30 more), but for the purpose of this exercise, that's all we need.

Most of the work happens in the ResourceActualDetail table it looks like this (with some columns deleted):

 

ResourceAllocationIdAllocationPercentageEndDateStartDate
79414/1/2018 0:002/25/2018 0:00
7951.153/4/2018 0:002/25/2018 0:00
7951.08753/11/2018 0:003/4/2018 0:00
79514/1/2018 0:003/11/2018 0:00
7961.053/4/2018 0:002/25/2018 0:00
7970.7253/4/2018 0:002/25/2018 0:00
153113/11/2018 0:002/25/2018 0:00
172114/1/2018 0:002/25/2018 0:00

 

 

Desired Output:

ResourceAllocationIdAllocationPercentageEndDateStartDateDate
79414/1/2018 0:002/25/2018 0:002/26/2018 0:00
79414/1/2018 0:002/25/2018 0:003/4/2018 0:00
79414/1/2018 0:002/25/2018 0:00
79414/1/2018 0:002/25/2018 0:004/1/2018 0:00

Notice the 7 day increments.

 

And the same for every ResourceAllocationID so that I can graph it like so:

lerecie_0-1617043849417.png

(error in Graph dates, should be with 7 day increments).

 

Where there is a different line for each Resource (connected through relationships to ResourceAllocation and ResourceActualDetail). Even if we can get it to a point where we can do that for each ResourceAllocationId I can figure out the rest 🙂

 

Current Approach:

These data tables are quite large already(~1M rows) , and my approach only has bloated that.

Trying to do everything as close to the source, this what I came up with:

 

Let

Source=PowerBI.Dataflows(null),

...

#"Changed Type" = Table.TransformColumnTypes(ResourceActualDetail1,{{"EndDate", type number}, {"StartDate", Int64.Type}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Dates", each List.Numbers([StartDate],([EndDate]-[StartDate])/7,7)),
#"Expanded Dates" = Table.ExpandListColumn(#"Added Custom", "Dates"),
#"Changed Type1" = Table.TransformColumnTypes(#"Expanded Dates",{{"Dates", type datetime}, {"EndDate", type datetime}, {"StartDate", type datetime}})
in
#"Changed Type1"

 

Essentially, I change the StartDate and EndDate fields to numeric Values and then I calculate the beginning of each week before i switch back all those values to Date type.

This part of the query doesn't fold, so I cannot take the advantage of the PowerFlow to do it. It also makes each refresh very slow (should be refresed multiple times a day).

 

Any suggestions or ideas?? Obviously looking for the most efficient way to achieve this so that it won't take many resources (I do have a premium instance).

 

Thanks!

 

2 REPLIES 2
lerecie
Frequent Visitor

I have tried both approaches and there's something that is not quite working.

 

I have focused in using your Value by End of the Month formula, since it looks the most complete for what I am trying to do, but it doesn't work.. i don't get the continuos dates and get breaks in the data still.

 

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

Power BI Carousel June 2024

Power BI Monthly Update - June 2024

Check out the June 2024 Power BI update to learn about new features.

PBI_Carousel_NL_June

Fabric Community Update - June 2024

Get the latest Fabric updates from Build 2024, key Skills Challenge voucher deadlines, top blogs, forum posts, and product ideas.