Forum Discussion
Not able to find Custom SQL option in Dataflow when source is databricks
- 1 year ago
Hi mhmmd_srf
When creating a dataflow in Power BI using Databricks as the data source, you may notice that the option to write custom SQL queries is not available, unlike with traditional relational sources like SQL Server or SAP HANA. This limitation exists because the Databricks connector in Power BI dataflows currently does not support query folding or direct SQL input within the Power Query Online editor. Instead, it enforces a more GUI-based approach where you must use the standard table and column selection method followed by transformation steps using Power Query (M language).
This can be frustrating if you rely on complex SQL logic or want to optimize performance by pushing query logic down to Databricks. One workaround is to define your transformations as views or custom queries within Databricks itself, and then reference those views from Power BI dataflows. Another alternative is to use a Power BI Desktop file to connect to Databricks, write your custom SQL there, and then export the M script to use inside the dataflow (though this still won’t enable direct SQL execution, it helps preserve logic). Until Microsoft adds support for native SQL editing for Databricks in dataflows, these workarounds are the only viable approaches.
- 1 year ago
Hi mhmmd_srf ,
I hope the response provided by the super users have addressed the issue. If yes please please mark the helpful reply or share your solution and accept it as solution, it will be helpful for other members of the community who have similar problems as yours to solve it faster.
Regards,
Chaithra.
Hi mhmmd_srf
When creating a dataflow in Power BI using Databricks as the data source, you may notice that the option to write custom SQL queries is not available, unlike with traditional relational sources like SQL Server or SAP HANA. This limitation exists because the Databricks connector in Power BI dataflows currently does not support query folding or direct SQL input within the Power Query Online editor. Instead, it enforces a more GUI-based approach where you must use the standard table and column selection method followed by transformation steps using Power Query (M language).
This can be frustrating if you rely on complex SQL logic or want to optimize performance by pushing query logic down to Databricks. One workaround is to define your transformations as views or custom queries within Databricks itself, and then reference those views from Power BI dataflows. Another alternative is to use a Power BI Desktop file to connect to Databricks, write your custom SQL there, and then export the M script to use inside the dataflow (though this still won’t enable direct SQL execution, it helps preserve logic). Until Microsoft adds support for native SQL editing for Databricks in dataflows, these workarounds are the only viable approaches.
hi Poojara_D12 thanks.. this is helpful.