Synapse Integration to VSCode not working (installation kernel & sparklighter ok)

Hi all,

I have some troubles to get the VSCode integration with synapse working.

Spark seems not to be recognized, other code is working fine (pandas, etc.).

Error when using spark commands:  [Error]: 'NoneType' object has no attribute 'read' (when trying to read csv.)

The things that are so far ok:

--> Installation ok with no errors in VSCode (conda, spark-lighter-lib, etc.), finished with ok in terminal

--> fabric-synapse-runtime-1-2 (Python 3.10.12) is visible and selected

--> all other integration functions in VSCode are working (workspase selection, publish and update ressource folder, update notebook, lankehouse files and tables, etc.

Only the spark lib makes trouble, but it seems it is an internal error, log messages for this seesion:

09:00:55,605 root INFO current_directory c:\Users\jda2ho\Documents\VisualStudioCode\FabricVSC\98c60a67-4e02-4fa6-908c-57de50976e22\SynapseNotebook\8fcf44b0-6fa0-4c53-91db-c4debf19801a\Notebook 1
09:00:55,605 root INFO workspace_path c:\Users\jda2ho\Documents\VisualStudioCode\FabricVSC\98c60a67-4e02-4fa6-908c-57de50976e22
09:00:55,605 root INFO log_path c:\Users\jda2ho\Documents\VisualStudioCode\FabricVSC\98c60a67-4e02-4fa6-908c-57de50976e22\logs\8fcf44b0-6fa0-4c53-91db-c4debf19801a
09:00:55,605 root INFO Using synapse remote kernel ...
09:00:55,605 root INFO Should attach session in dev mode False
09:00:56,60 root INFO Starting session e05151e2-26ec-406c-93b1-bd9510a7b9d3...
09:00:57,787 root INFO Getting refresh token...
09:00:58,934 root INFO https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxx.pbidedicated.windows.net/webapi/capacities/C4AA2FF0-BC0F-4B1D-A228-DCCCAAA18CFC/workloads/Notebook/Data/Direct/sparklighter/api/tjs/versions/2022-04-30/artifacts/8fcf44b0-6fa0-4c53-91db-c4debf19801a/sessions/e05151e2-26ec-406c-93b1-bd9510a7b9d3
09:01:01,379 root INFO <session_management.SessionStatus object at 0x00000247CB7B9C30>
09:01:01,379 root ERROR Failed to initialize Spark Lighter variables. Create session failed {'code': 'InternalError', 'subCode': 0, 'message': 'An internal error occurred.', 'timeStamp': '2024-10-02T07:01:01.4472304Z', 'httpStatusCode': 500, 'hresult': -2147467259, 'details': [{'code': 'RootActivityId', 'message': 'd7fcbe3d-98af-4329-b52a-b260dc20ea9b'}, {'code': 'Param1', 'message': 'Response status code does not indicate success: 430 ().'}]}
09:01:01,380 root INFO Registering Spark Lighter magics for IPython...
09:01:01,380 root INFO Registered Spark Lighter magics for IPython.

Any help would be appriciated!

Best regards,

Daniel