Forum Discussion
Column profiling not using query folding on Databricks
fmdus you are asking it to analyze all rows. That cannot be folded. The column profiling is a feature of Power Query. To do the analysis, it must retrieve all records to do the analysis. It is like loading the data. At some point, it has to finally retrieve the full recordset. All query folding does is puts that off until the last possible moment when either folding breaks, or the data loads, or you ask the tool to do something that cannot be folded, like do a column profile on the entire column.
Theoretically could it generate multiple SQL statements just to show the column quality and profiling of the recordset? Yes. And theortically, that would hammer the SQL Server with a ton of queries. I suppose you could post that as an Idea in the Ideas forum, but remember, Power Query is a transformation tool, not an analysis tool. At some point, do the analysis in DAX.