Forum Discussion
Custom Function to filter a table (1 Query) multiple times
- Anonymous4 years ago
You can do something like this.
First, load your table from the database, and then add your main filter that you want to flow through all queries. Note that I needed to transform the date columns with the locale; you may not need that step:
let
Source = DBSource,
#"Changed Type with Locale" = Table.TransformColumnTypes(Source, {{"Date", type date}}, "en-GB"),
#"Changed Type" = Table.TransformColumnTypes(#"Changed Type with Locale",{{"ID", Int64.Type}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each [Date] >= #date(2020, 1, 7) and [Date] <= #date(2021, 1, 4))
in
#"Filtered Rows"Now you have this table (I named the query "Data"):
ID Date
1 7/5/2020 2 9/3/2020 3 11/2/2020 4 1/1/2021 Now you can add another query that will first buffer the filtered table, then repeat the table 30 times, and then split the table into 30 nested tables. Then add an index to the table so you have a way to delineate each table. This Query is names Tables:
let
Source = Data,
BufferTable = Table.Buffer(Source),
RepeatTable = Table.Repeat(BufferTable, 30),
SplitTable = Table.Split(RepeatTable, List.Max(RepeatTable[ID])),
#"Converted to Table" = Table.FromList(SplitTable, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Renamed Columns" = Table.RenameColumns(#"Converted to Table",{{"Column1", "Tables"}}),
#"Added Index" = Table.AddIndexColumn(#"Renamed Columns", "Index", 0, 1, Int64.Type),
#"Reordered Columns" = Table.ReorderColumns(#"Added Index",{"Index", "Tables"})
in
#"Reordered Columns"Now you have 30 tables, each filtered already via the data table. By buffering the table first, we ensure that that table is not reevaluated for the duration of THIS query--which means that you'll need to make a decision at this point. If you are comfortable working with nested tables, then you can make all of your transforms right in this query, and continue to enjoy the benefit of the single buffered table. If you'd rather work with each table on its own, you can either add new blank queries with the source being:
=Tables{4}//Just the name of the query (Tables) and the row number--Tables{number}
Which is the 5th table in the Tables Query.
You can also just right click on a table in the list and choose "Add as new query."
Again, the buffered table only exists through the duration of the Tables query. Even so, it's probably still going to be a lot faster than 30 separate queries--but that's for you to decide!
---Nate
You can do something like this.
First, load your table from the database, and then add your main filter that you want to flow through all queries. Note that I needed to transform the date columns with the locale; you may not need that step:
let
Source = DBSource,
#"Changed Type with Locale" = Table.TransformColumnTypes(Source, {{"Date", type date}}, "en-GB"),
#"Changed Type" = Table.TransformColumnTypes(#"Changed Type with Locale",{{"ID", Int64.Type}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each [Date] >= #date(2020, 1, 7) and [Date] <= #date(2021, 1, 4))
in
#"Filtered Rows"
Now you have this table (I named the query "Data"):
ID Date
| 1 | 7/5/2020 |
| 2 | 9/3/2020 |
| 3 | 11/2/2020 |
| 4 | 1/1/2021 |
Now you can add another query that will first buffer the filtered table, then repeat the table 30 times, and then split the table into 30 nested tables. Then add an index to the table so you have a way to delineate each table. This Query is names Tables:
let
Source = Data,
BufferTable = Table.Buffer(Source),
RepeatTable = Table.Repeat(BufferTable, 30),
SplitTable = Table.Split(RepeatTable, List.Max(RepeatTable[ID])),
#"Converted to Table" = Table.FromList(SplitTable, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Renamed Columns" = Table.RenameColumns(#"Converted to Table",{{"Column1", "Tables"}}),
#"Added Index" = Table.AddIndexColumn(#"Renamed Columns", "Index", 0, 1, Int64.Type),
#"Reordered Columns" = Table.ReorderColumns(#"Added Index",{"Index", "Tables"})
in
#"Reordered Columns"
Now you have 30 tables, each filtered already via the data table. By buffering the table first, we ensure that that table is not reevaluated for the duration of THIS query--which means that you'll need to make a decision at this point. If you are comfortable working with nested tables, then you can make all of your transforms right in this query, and continue to enjoy the benefit of the single buffered table. If you'd rather work with each table on its own, you can either add new blank queries with the source being:
=Tables{4}//Just the name of the query (Tables) and the row number--Tables{number}
Which is the 5th table in the Tables Query.
You can also just right click on a table in the list and choose "Add as new query."
Again, the buffered table only exists through the duration of the Tables query. Even so, it's probably still going to be a lot faster than 30 separate queries--but that's for you to decide!
---Nate
Hi Anonymous,
This sounds about exactly what I need. I really appreciate you providing me this solution. I'll test this approach and will let you know by Saturday 12/02.
You totally understood what I needed this for, just using your head which is very very impressive.
After I pass on the date filters, I needed to create those 30 tables for 6 different regions. (5 tables per region). These tables would then be used to interact with user input to perform what ifs, (5 scenarios per region). One refresh after user inputs and we get 5 scenarios for each region, which is then visualized in Power BI. Meaning I would very likely work with each tables on its own after this point as the user input reference for each tables will be different.
Can't wait to try it.
Thank you very much.