Forum Discussion
SQL Code or Power Query for data massage
Generally you will have better performance the further up the chain that you push things. Think of it this way, if you implement the data cleanup in SQL you are pushing the work to the SQL box and thus using the server's resources to execute on the code versus with Power Query, a lot of the work is being done by the computer on which you are running Power BI. Generally going to be faster executing on a server which likely has better CPU/RAM than a computer. In addition, the query and cleanup is essentially happening locally so faster disk I/O. Plus, you are only pushing the relevant data across the wire (network) so that could very likely be much faster as well.
Again, these are generalities and your specific situation may be different.
In general, it is best to push the data massaging back in the data pipeline as far as possible. So, if you have the SQL skills, do it there. In fact, write a SQL View and then just connect your query to that. The data wrangling and other features in Power BI are for the folks that do not have the technical ability or access to be able to do direct data manipulation within the source system(s).
Greg_Deckler Understood but is there a performance difference between the 2 ways?
Generally you will have better performance the further up the chain that you push things. Think of it this way, if you implement the data cleanup in SQL you are pushing the work to the SQL box and thus using the server's resources to execute on the code versus with Power Query, a lot of the work is being done by the computer on which you are running Power BI. Generally going to be faster executing on a server which likely has better CPU/RAM than a computer. In addition, the query and cleanup is essentially happening locally so faster disk I/O. Plus, you are only pushing the relevant data across the wire (network) so that could very likely be much faster as well.
Again, these are generalities and your specific situation may be different.