Forum Discussion
Community ChampionPower Bi Challenge - Round 2
Community ChampionLooks like you are the only one who has definitely hit the point by utilizing TOCSV to handle this situation. Your genius solution deserves, with no doubt, to be marked as the accepted solution for this challenge.
However, I’m not going to make it easy for you 😁. Your solution works like magic in your file, but seems to have a little trouble in my file. Looks like someone has changed the order of the columns or someone has added another attribute column like “System ID”, “Sensor Location” or just “Row ID”. Not sure who did that 🤣
Community Championtamerj1 Yeah, the order of the columns is important or at least you need to know the order in terms of which column goes where. Would need to know specifics of your file to come up with a specific solution but will try giving a general solution a shot. But, for example, if you just added an additional column of information to the "front" then you could change your GENERATESERIES from 3 to 146 to 4 to 147. I have some ideas on a general "can't fail" solution but need to test it out.
- Greg_Deckler
tamerj1 Hey, here's a version that doesn't require hardcoding and you can theoretically add as many pre-columns as you want. Basically Measure 2 but with a dynamic finding of the number of commas before the first value column.
Measure 4 = VAR __Text = SUBSTITUTE( TOCSV( 'T_MEASURES', , ",", 1 ), UNICHAR(10), "|" ) VAR __1 = PATHITEM( __Text, 1 ) VAR __FirstColon = FIND(":", __1) VAR __Left = LEFT( __1, __FirstColon ) VAR __Commas = LEN( __Left ) - LEN( SUBSTITUTE( __Left, ",", "" ) ) VAR __2 = PATHITEM( __Text, 2 ) VAR __3 = SUBSTITUTE( __2, ",", "|", __Commas ) VAR __4 = PATHITEM( __3, 2 ) VAR __Path = SUBSTITUTE( __4, ",", "|" ) VAR __Table = ADDCOLUMNS( GENERATESERIES(1, 144, 1), "__Value", PATHITEM( __Path, [Value]) + 0 ) VAR __Result = AVERAGEX( __Table, [__Value] ) RETURN __Result - Greg_Deckler
tamerj1 OK, one "final" version. This one *should* work with multiple rows of sensor data on the same day and doesn't assume basically anything in terms of how many columns of data, just that the first column of data that the name includes a ":" and that after that the data columns are sequential with no non-data columns mixed in.
I don't have the original data file so I can't actually test it with multiple data on the same day (although that doesn't make sense in this scenario it might make sense in other similar circumstances or scenarios so I wanted to try doing it).
Measure 5 = // Deal with the header to figure out what columns need to be removed VAR __TableWHeaders = SUBSTITUTE( TOCSV( 'T_MEASURES', , ",", 1 ), UNICHAR(10), "|" ) // TOCSV with Headers VAR __Headers = PATHITEM( __TableWHeaders, 1 ) //headers VAR __FirstColon = FIND(":", __Headers) // find the first colon, which should be the first column of data VAR __Left = LEFT( __Headers, __FirstColon ) // grab everything to the left of the first colon VAR __Commas = LEN( __Left ) - LEN( SUBSTITUTE( __Left, ",", "" ) ) // figure out how many commas (columns) are in the "pre-columns" before the data // Now deal with the data which might have multiple rows VAR __TableWOHeaders = SUBSTITUTE( TOCSV( 'T_MEASURES', , ",", 0 ), UNICHAR(10), "|" ) // TOCSV without Headers VAR __Count = COUNTROWS('T_MEASURES') // number of rows in table VAR __Data = // Generate a table with a column that strips out the pre-columns ADDCOLUMNS( GENERATESERIES(1, __Count, 1), "__Data", VAR __Text = PATHITEM( __TableWOHeaders, [Value]) VAR __Path = SUBSTITUTE( __Text, ",", "|", __Commas ) VAR __Result = PATHITEM( __Path, 2 ) RETURN __Result ) VAR __DataText = CONCATENATEX( __Data, [__Data], "|" ) // convert the table back into a text string with each row as a path item VAR __Path = SUBSTITUTE( __DataText, ",", "|" ) // convert all columns to path items VAR __DataColumns = LEN( __DataText) - LEN( SUBSTITUTE( __DataText, ",", "" ) ) // figure out how many columns of data we are dealing with VAR __Table = // Convert the path into a table ADDCOLUMNS( GENERATESERIES(1, __DataColumns, 1), "__Value", PATHITEM( __Path, [Value]) + 0 ) VAR __Result = AVERAGEX( __Table, [__Value] ) // run an aggregation against the table RETURN __Result - tamerj1
True, but I was hoping not to have to change the code. Does that sound greedy? Cause I am greedy 🤑
- Greg_Deckler
tamerj1 Yeah, the problem is that there isn't a SELECTCOLUMNSEXCEPT function that would select all the columns except those that you specify. If you had that function, even still you would have to modify the code if you add an additional column. The only way to achieve a no code revisions would be if you hard coded all of the desired columns into a SELECTCOLUMNS statement or if you could come up with a way to SELECTCOLUMNSIN function where you could pass a table and if the column name appeared in a row value then it would be selected otherwise not. It is possible to create your time table, one is included in the PBIX but I couldn't pull off the last piece of it where I used that as selection criteria for columns.
I came up with a couple additional approaches to lessen the burden on modifying the code. PBIX is attached as well. Also, these approaches seem to make the code more stable across charts types in my testing.
This one abuses the PATH functions even more and uses SUBSTITUTE to find the 2nd comma and replace it. All you need to do then is if you add an additional column to the front then you modify the instance of the comma you are replacing. Note that this version is limited to when you will only ever have a single row of data that you are averaging.
Measure 2 = VAR __Text = SUBSTITUTE( TOCSV( 'T_MEASURES', , ",", 1 ), UNICHAR(10), "|" ) VAR __1 = PATHITEM( __Text, 1 ) VAR __2 = PATHITEM( __Text, 2 ) VAR __3 = SUBSTITUTE( __2, ",", "|", 2 ) VAR __4 = PATHITEM( __3, 2 ) VAR __Path = SUBSTITUTE( __4, ",", "|" ) VAR __Table = ADDCOLUMNS( GENERATESERIES(1, 144, 1), "__Value", PATHITEM( __Path, [Value]) + 0 ) VAR __Result = AVERAGEX( __Table, [__Value] ) RETURN __ResultThis version attempts to ensure that all "pre" columns are moved to the end prior to grabbing the data. So, if you add a "pre" column, you will need to add an additional DISTINCT and correpsonding SUBSTITUTEWITHINDEX. The index is added to the end of the table so once this is done you can just grab items 1 - 144 every time. I tried multiple ways of trying to eliminate multiple columns at a time using this approach but the function refused to cooperate.
Measure 3 = VAR __Table = 'T_MEASURES' VAR __Sensors = DISTINCT('T_MEASURES'[Sensor Tag]) VAR __Dates = DISTINCT('T_MEASURES'[Date]) VAR __Table1 = SUBSTITUTEWITHINDEX( __Table, "Index", __Sensors, [Sensor Tag], ASC) VAR __Table2 = SUBSTITUTEWITHINDEX( __Table1, "Index1", __Dates, [Date], ASC) VAR __Text = SUBSTITUTE( TOCSV(__Table2,,,0), ",", "|" ) VAR __Table3 = ADDCOLUMNS( GENERATESERIES(1, 144, 1), "__Value", PATHITEM( __Text, [Value]) + 0 ) VAR __Result = AVERAGEX( __Table3, [__Value] ) RETURN __Result - tamerj1
I don't have enough words to describe my happiness while reading this. I really didn't expect to learn something out of this challenge but indeed you you are one of best teachers I have learned from. The way you approach the problem made me speachless. That opened my eye to many ideas that can realize new methods of solving problems. I realized that I still have so much to learn.
You are right, at some point you have to hardcode something. True that the column order might change and true that more information columns can be added but tye number of value columns and their NAMES remain unchanged. TOCSV ofers more than the values of the columns.
- AlexisOlson
Super User
Greg_Deckler, just FYI, your Measure 5 is a bit off. I'd guess some off-by-one bug somewhere.
I'm attaching a pbix that has the data baked in (rather than referencing an Excel file) in case you or others want to tinker with the source data. I used my custom Table.ToM function to produce this (and discovered I hadn't accounted for column names that need to be escaped with a #" " wrapper, so I'll need to update my function to handle that).
