Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Enhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.

Databricks connector not escaping single quotes in Direct Query

I am running into an issue when using direct query with Databricks SQL Warehouses using the Azure Databricks Connector. I have text fields with single quote characthers in the string that need to be rendered in the report. When filtering for a value with an apostraphe the query returns null. Looking at the query history in databricks it looks like the SQL query being generated is using a second single quote to escape the apostraphe, but this is not the escape charachter in Databricks SQL.  

Unless the string is prefixed with r, use \ to escape special characters (e.g. ' or \).

If the string is prefixed with r there is no escape character.

Here is documentation for string-type in Databricks SQL.  

 

 

 

select
sum(`sales_mlc_amt`) as `C1`
from
(
select
`OTBL`.`sales_amt`
from
`hive_metastore`.`database`.`table` as `OTBL`
left outer join `hive_metastore`.`table`.`market_dim` as `ITBL` on (`OTBL`.`mkt_skid` = `ITBL`.`mkt_skid`)
where
`ITBL`.`mkt_name` = 'Market''s Territory'
) as `ITBL`

 Thanks!

Status: Investigating
Comments
Anonymous
Not applicable

Hi @Matt_Morscher ,

 

Here is a similar thread about using DirectQuery in Azure Databricks. Azure databricks with DirectQuery isn't listed in the supported source in the document . 

 

Best Regards,
Community Support Team _ Caitlyn

Matt_Morscher
New Member

@Anonymous I think that issue is seperate. This user looks like they were not using the Azure Databricks connector which is the one I am using. Here is the link to that connector. https://learn.microsoft.com/en-us/azure/databricks/partners/bi/power-bi

 

Looking at the supported data source list in the document you linked. I do see Azure Databricks listed as supporting Direct Query. 

Data source Connect from DesktopConnect and refresh from serviceDirectQuery / Live connectionGateway (supported)Gateway (required)Power BI Dataflows
Azure DatabricksYesYesYesYesNoYes

 

I believe that the issue is that when the connector builds the query string it is using an escape charachter that is not supported in Databricks SQL. 

Thanks for looking into it!