Forum Discussion
Compute effective approach; sumx, summarize, sum - case of transactional sales data and exch. rates
Dear community,
Main issue; finding the most compute effective approach.
Current Datamodel;
Background info;
- Sales Revenue and Cost is transactional data, each line representing information about actual sales at a particular date.
- Sales Revenue lines are in different currencies.
- Table has approx. 11 million lines.
- Exchange Rates contains information about Currency Exchange Rates from and to currency at particular dates. Date is a "valid from date", and is not consistent in terms of an exchange rate for all dates in Sales Revenue and Cost (most commonly only last date of each month).
Task;
- Establish a consolidated report in a consolidate currency by choice of the user.
- Use latest valid currency exchange rate "before" Sales Revenue and Cost" Date.
My first approach;
- Solution; Using Power Query with Table.SelectRows to add columns to Sales Revenue... table with selected reporting currencies and calculated amount based on that. I.e. perform the transformation at the time of loading the data.
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each Table.Max(Table.SelectRows(Rates, (Rate) => Rate[Date]<=[Date] and Rate[Currency]=[Currency]),"Date")
),
#"Expanded Custom" = Table.ExpandRecordColumn(#"Added Custom", "Custom", {"Rate"}, {"Rate"})
- Problem; with a clean load Power BI services could load around 100k lines per second, with above solution it was reduced to only 10 lines per second and with 11 million lines it would take days to finish.
Second approach;
- Solution; Using DAX for the transformation via pure SUMX(Sales Revenue...) with calculate conditions for the conversion
Sales Revenue =
SUMX(
'Sales Revenue and Cost',
'Sales Revenue and Cost'[Line Amount] *
Calculate(
min('Exchange Rates'[Rate]),
filter('Exchange Rates',
'Exchange Rates'[From Currency]='Sales Revenue and Cost'[Company Currency] &&
'Exchange Rates'[Date]<='Sales Revenue and Cost'[Date] &&
'Exchange Rates'[To Currency]=SELECTEDVALUE('Reporting Currency'[Currency])
)
)
)
- Problem; Solutions provides the perfect answer and best feature of being dynamically able to select "another" reporting currency, however, publishing this to services it is evident that it is very ineffective, slow and at times power bi services post an error stating too much memory consumption - i.e. not a "production able" solution.
Third approach, and where I am stuck:
- Solution; I've read that SUMX is not to be used like I did cause it does a calculation for every single row every time, and that people have come around it by using SUMX VALUES, summarize or a combination. I am at the point of trying out SUMX VALUES but can't get it to play out correct and am wondering if it is the right solution at all, 2 measures combined to "Sales Revenue";
Sales Revenue = SUMX( VALUES('Sales Revenue and Cost'[Currency]), [Sales Revenue LCY] * [Sales Revenue Reporting Currency Rate] ) Sales Revenue LCY = SUM('Sales Revenue and Cost'[Line Amount]) Sales Revenue Reporting Currency Rate = Calculate( min('Exchange Rates'[Rate]), filter('Exchange Rates', 'Exchange Rates'[From Currency]=MAX('Sales Revenue and Cost'[Currency]) && 'Exchange Rates'[Date]<=MAX('Sales Revenue and Cost'[Date]) && 'Exchange Rates'[To Currency]=SELECTEDVALUE('Reporting Currency'[Currency]) ) )- Also I am concerned if the method secures that I am using the correct currency exchange rate "before" the Sales Revenue posting date - as a minimum I would like that a monthly level of summarized amount it is translated by the exchange rate "before".
I am stuck, quite beginners level and have spent so many hours to the point of much frustration - any hint of the right direction would be greatly appreciated!
Best regards,
Lasse
11 Replies
- LasseL
Helper I
I continued a bit on my own.
Current trial;
Sales Revenue = VAR ExchangeRate = Calculate( min('Exchange Rates'[Rate]), filter('Exchange Rates', 'Exchange Rates'[From Currency]=MIN('Sales Revenue and Cost'[Currency]) && 'Exchange Rates'[Date]<=MAX('Sales Revenue and Cost'[Date]) && 'Exchange Rates'[To Currency]=SELECTEDVALUE('Reporting Currency'[Currency]) ) ) RETURN SUMX( SUMMARIZE('Sales Revenue and Cost','Sales Revenue and Cost'[Currency]), [Sales Revenue LCY] * ExchangeRate )With [Sales Revenue LCY] being;
Sales Revenue LCY = SUM('Sales Revenue and Cost'[Line Amount])It seems to perform better compute and memorize-wise, and a visual with Year-Month calculates correct Reporting Currency Amount but totals the columns incorrectly with total sum of LCY times "Last Exchange Rate"...
And I still have my doubts if this is best practice at all 😞
- v-yingjl
Community Support
Hi LasseL ,
Based on your description, the current trial will have a better performance beacuse it uses variable and summazire a table in the sumx() function.
"but totals the columns incorrectly with total sum of LCY times "Last Exchange Rate""
What is your expected output? Could you please consider sharing more details like sample file, screenshots etc. about this issue for further discussion?
Sample data and expected output would help tremendously.
Please see this post regarding How to Get Your Question Answered Quickly:
https://community.powerbi.com/t5/Community-Blog/How-to-Get-Your-Question-Answered-Quickly/ba-p/38490Best Regards,
Yingjie Li
- LasseL
Helper I
Dear Yingjie,
Thank you so much for getting back.
For confidential reasons I can't share the actual PBIX and data that I am working in, but I tried to reproduce a demo dataset (excel) and a PBIX fil with similar setup.
Attached;
- PBIX:
https://dbizdk.sharepoint.com/:u:/s/dbizApS/EUMQiIGSK8FPow-lVdwdYTYBlcqcfL80iQSwAmtKIyU92Q?e=VgX1KV - Excel sources:
https://dbizdk.sharepoint.com/:x:/s/dbizApS/Ec22y1l-VhVLqxrYHe9j7ogB_ZQi0cZOOhOkZzMGvrX4ag?e=5np5Z5 - Update, download link replaced by wetransfer: https://we.tl/t-264shFE8H6
What I would expect is the totals of Sales to be correct with latest exchange rate when choosing either DKK or EUR as a reporting currency;
Does this help?
- PBIX: