Getting random characters after notebook metadata modification using Rest API

[chetanhiwale]

I am trying to modify notebook metadata using fabric rest api. After decoding the notebook content we get attached item ( lakehouse , warehouse , env ) and cells. The cells metadata is getting added with some extra back slashed and '\\n' characters. 

My assumption is fabric is using these extra characters to detect cells, new lines and indentation. Is issue is coming when we use '\n' character in code it just breaks to next line. 

Original code : 

print('abc')

# Chetan count(b'\ n') ok world

str_ = ''

str_ = re.sub(r"[\\\'", '\n' , str_)

---------------------------------------------
Adding sample metadata of cell, 
['\\n\\n# CELL ********************\\n\\nprint(\\\'abc\\\')\\n\\n# Chetan count(b\\\'\\\\n\\\') ok world\\nstr_ = \\\'\\\'\\nstr_ = re.sub(r"[\\\\\\\\\\\\\\\'", "\\\\n", str_)\\n\\n# METADATA ********************\\n\\n# META {\\n# META "language": "python",\\n# META "language_group": "synapse_pyspark"\\n', "\\n'"]

---------------------------------------------

After updating the defination of notebook using REST api endpoint,

print('abc')

# Chetan count(b'\

') ok world

str_ = ''

str_ = re.sub(r"[\\\\'", "\

", str_)

The issue is coming for commented part and re.sub() line. It added extra backslash and break to next. How can solve this issue. I tried to use replace function for extra backslashes but doesnt work for all conditions. 

[V-yubandi-msft]

Hello @chetanhiwale ,

We noticed we haven't received a response from you yet, so we wanted to follow up and ensure the solution we provided addressed your issue. If you require any further assistance or have additional questions, please let us know.

Your feedback is valuable to us, and we look forward to hearing from you soon

[V-yubandi-msft]

Hello @chetanhiwale ,

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution we provided for your issue worked for you  or let us know if you need any further assistance?

Your feedback is important to us, Looking forward to your response. 

Thank You.

[V-yubandi-msft]

Hi @chetanhiwale ,

We wanted to reach out again as we haven't received any updates from you. Did the solution we provided help resolve your issue? If you still need assistance or have any additional questions, please don't hesitate to let us know.
If the issue is resolved, please mark it as the Accepted Solution. This will help others find the solution more easily.

Thank You.

[chetanhiwale]

Thanks @V-yubandi-msft , thanks for the response. Here I decoding the payload with base64 which return string. After these extra characters are being added. Is there any way to remove the characters. 

[V-yubandi-msft]

Hi @chetanhiwale ,

Thanks for the details. It seems the issue is happening during the decoding process, where extra escape characters (\ and \n) are being added to the notebook content.

Here’s a step-by-step approach to fix it.

• Ensure the base64-decoded string is handled correctly. Sometimes, decoding can introduce extra escape characters, especially if the payload is treated as a raw string. Use Python's base64 library to decode the payload and then process it as a string.

• After decoding, clean up the extra escape characters by replacing them appropriately. For instance, you can use Python's replace() function to handle \\n and \\.

• This will ensure the newlines and backslashes are correctly formatted.

• Before sending the updated content back via the REST API, make sure the payload is re-encoded properly. Use the base64 library to encode the cleaned payload.

• After making these changes, test the updated payload by sending it through the REST API and verify that the notebook content is correctly formatted without extra escape characters.

You can try these steps and let me know if you need any additional details.

If my response solved your query, please mark it as the Accepted solution to help others find it easily.

[chetanhiwale]

Hello @V-yubandi-msft , thanks for your input. I am using base64 for encoding and decoding the notebook defination. I have replace the extra characters like \\\' , \\\\\\\ with ' and \\. I am not able to handle \n condition.After updating the notebook , the \n is consider as next line. Simple example can be
print("microsoft \n fabric"),

after updating the notebook it becomes,
print("microsoft \
fabric")
Please share thoughts on this !!

[V-yubandi-msft · Accepted solution]

Hi @chetanhiwale ,

Thank you for your feedback. We attempted using the example you mentioned: print ("microsoft \n fabric").

As you noted, by default, \n in Python represents a newline character. To display \n as plain text, additional steps are required.

We need to escape the backslash (\) or use a raw string. Here are the steps:

1. Escape the backslash (\\n)

print ("microsoft \\n fabric")

2. Use a raw string (r"string")

• \\n: The double backslash \\ instructs Python to interpret \n as a literal \n instead of a newline character.
• r"...": The r prefix before a string denotes a raw string, which treats backslashes as literal characters.

If the suggestions provided do not resolve your issue, kindly share a snippet of the exact code you are using for encoding, decoding, and updating the notebook. With more context, I can assist further in debugging.

If my response solved your query, please mark it as the Accepted solution to help others find it easily.

And if my answer was helpful, I'd really appreciate a 'Kudos'.

[V-yubandi-msft]

Hi @chetanhiwale ,

Thanks for reaching out to the Microsoft Fabric Community.  It looks like the issue you're facing is related to JSON encoding and escape sequence handling when modifying notebook metadata via the Fabric REST API.

Here are some recommended steps to resolve it.

1. Ensure the JSON response is correctly decoded before making any changes. This prevents issues like double escaping of backslashes (\\) and incorrect newline (\n) formatting.

2. When modifying the content and sending it back through the API, check that JSON encoding doesn't add extra escape sequences. Some serialization methods might automatically add extra backslashes, so double-check your update request.

3. After updating the notebook, fetch the content again to see if there are any unintended formatting changes. If needed, clean up any extra backslashes before sending the update.

4. If backslashes are still being incorrectly added, review how JSON formatting is being done and adjust how escape characters are handled. Be careful when replacing backslashes, as some are necessary for valid syntax.
   
   

If my response solved your query, please mark it as the Accepted solution to help others find it easily.  And if my answer was helpful, I'd really appreciate a 'Kudos'.