Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
Manuel_Grutsch
New Member

Escape character vanishes on load. JSON field. Databricks connector

Hi everyone

 

We access a table containing a JSON column with the Power BI connector for SQL Warehouse cluster in Databricks.

 

The field looks like this in Databricks (complex datatype):

 

 

[{"key":"FOO","label":"FOOBAR","value":"ABC 54\""},{...}]

 

 

 

It also looks like that when we connect to the same SQL Warehouse cluster with Python.

 

However, in Power BI it looks like this (string). The escape character \ is missing:

 

 

[{"key":"FOO","label":"FOOBAR","value":"ABC 54""},{...}]

 

 

 

 

This leads to an error when we try to parse this JSON.

There are no other steps done in Power Query. That is just how the data comes in.

 

Could this be a bug? Some kind of parsing on the way?

 

Thanks for the help!

Kind regards

Manuel

2 REPLIES 2
Manuel_Grutsch
New Member

Hi @v-stephen-msft

 

Thanks for the reply.

 

Your first suggestion sadly won't work, because the fields we are dealing with a custom fields in SAP. New ones can be added and the are barely any restriction for neither label or value. In other words, at any point there can be double quotes in the strings. Therefore we have no reliable way to do the replace value without messing with the required double quotes of the JSON structure.

 

As for your second suggestion, how exactly would you go about validating and correcting the JSON string with a custom column and M?

 

Thanks and beste regards

Manuel

v-stephen-msft
Community Support
Community Support

Hi @Manuel_Grutsch ,

 

After loading the data into Power BI, use the Power Query Editor to manually insert the missing escape characters. You can use the Replace Values feature to find all instances of 54"" and replace them with 54\"".

Or before attempting to parse the JSON in Power BI, use a custom column in Power Query to validate and possibly correct the JSON string. This can be done using the M language to ensure that the JSON string is correctly formatted for parsing.

 

Best Regards,

Stephen Tao

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

Power BI Carousel June 2024

Power BI Monthly Update - June 2024

Check out the June 2024 Power BI update to learn about new features.

RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

Top Solution Authors
Top Kudoed Authors