Forum Discussion
Dynamically select column based on slicer for regression/correlation calculation
Hi Anonymous
Here I'm using Field Parameters for dynamic selection of the X-Axis and Y-Axis. However, when trying to calculate the slope and intersection dynamically I hit the wall :-DI have a strong feeling that there should be a much simpler way to accomplish the same but this is what I was able to accomplish so far.
Worth mentioning that with hundreds of columns and thousands of rows this solution might not be feasible in terms of performance.
Also note that I don't have very strong visualization skills and I didn't know how how to add the manual trend line to the scattered plot so I just overlapped it with a line chart. Please do not laugh at that 🙂
Greg_Deckler Appreciate your input on this. Was that over thinking or it is just what it is?
Regression Value =
VAR XColumn = MAX ( 'X-Axis'[Parameter] )
VAR YColumn = MAX ( 'Y_Axis'[Parameter] )
VAR T = ALLSELECTED ( 'Table' )
VAR NumberOfRows = COUNTROWS ( T )
VAR String1 = TOCSV ( T, -1, ",", TRUE )
VAR Items1 = SUBSTITUTE ( String1, UNICHAR ( 10 ), "|" )
VAR T1 = GENERATESERIES ( 1, NumberOfRows, 1 )
VAR T2 = SELECTCOLUMNS ( T1, "@Headers", PATHITEM ( Items1, 1 ), "@Details", PATHITEM ( Items1, [Value] + 1 ) )
VAR T3 =
GENERATE (
T2,
VAR HeaderString = [@Headers]
VAR HeaderItems = SUBSTITUTE ( HeaderString, ",", "|" )
VAR DetailString = [@Details]
VAR DetailItems = SUBSTITUTE ( DetailString, ",", "|" )
VAR Length = PATHLENGTH ( HeaderItems )
VAR T4 = GENERATESERIES ( 1, Length, 1 )
VAR T5 = SELECTCOLUMNS ( T4, "@Header1", PATHITEM ( HeaderItems, [Value] ), "@Detail1", PATHITEM ( DetailItems, [Value] ) )
VAR T6 = SELECTCOLUMNS ( T4, "@Header2", PATHITEM ( HeaderItems, [Value] ), "@Detail2", PATHITEM ( DetailItems, [Value] ) )
VAR T7 = FILTER ( T5, [@Header1] = "'Table'[" & XColumn & "]" )
VAR T8 = FILTER ( T6, [@Header2] = "'Table'[" & YColumn & "]" )
RETURN
CROSSJOIN ( T7, T8 )
)
VAR LinestTable = LINESTX ( T3, [@Detail2], [@Detail1] )
VAR Slope = MAXX ( LinestTable, [Slope1] )
VAR Intercept = MAXX ( LinestTable, [Intercept] )
VAR X =
MAXX (
'Table',
VAR T1 = CALCULATETABLE ( 'Table' )
VAR String1 = TOCSV ( T1, 1, ",", TRUE )
VAR Items1 = SUBSTITUTE ( String1, UNICHAR ( 10 ), "|" )
VAR T2 = SELECTCOLUMNS ( { ( 1, 2 ) }, "@Headers", PATHITEM ( Items1, 1 ), "@Details", PATHITEM ( Items1, 2 ) )
VAR T3 =
GENERATE (
T2,
VAR HeaderString = [@Headers]
VAR HeaderItems = SUBSTITUTE ( HeaderString, ",", "|" )
VAR DetailString = [@Details]
VAR DetailItems = SUBSTITUTE ( DetailString, ",", "|" )
VAR Length = PATHLENGTH ( HeaderItems )
VAR T4 = GENERATESERIES ( 1, Length, 1 )
RETURN
FILTER (
SELECTCOLUMNS (
T4,
"@Header", PATHITEM ( HeaderItems, [Value] ),
"@Detail", PATHITEM ( DetailItems, [Value] )
),
[@Header] = "'Table'[" & XColumn & "]"
)
)
RETURN
MAXX ( T3, VALUE ( [@Detail] ) )
)
RETURN
Slope * X + Intercept
tamerj1 I'll have to take a look at this in greater detail. Been a while since I played around with regression analysis and I haven't really looked much at LINEST function. Thanks for the PBIX. I'll have a look at what you did here, it looks pretty interesting.
I do come back to that even with 400 column and let's say 10,000 rows, that's only 4 million rows unpivoted so shouldn't be that problematic and shouldn't increase the size of the dataset that tremendously since there should be pretty decent compression. Would likely simplify this problem greatly. Always up for a challenge though!