Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Get certified in Microsoft Fabric—for free! For a limited time, get a free DP-600 exam voucher to use by the end of 2024. Register now

Reply
SnackSeeker91
Regular Visitor

VScode Synapse PROXY

Hello,

Could someone tell me how to set a proxy for PySpark Lighter? I have everything installed correctly (VSCode, Synapse kernels, Python, Miniconda3), but I can't bypass the network settings with the proxy in PySpark Lighter. With the proxy set in VSCode settings, I can see tables, download files, and download notebooks, but I can't use the notebook via Synapse.

 

I know I have to set the proxy there because when I make an HTTP request without the proxy, I see a similar error, HTTPSConnectionPool. With the proxy settings, I can even manually ping the SparkLighter server but with the wrong response of course.

 

I have another Synapse Fabric that is working well because there is no firm VPN but in this case i can work only with firm VPN.

 

Thank you

 

SnackSeeker91_0-1729680693502.png

 



11:30:23,724 root INFO Using synapse remote kernel ...
11:30:23,724 root INFO Should attach session in dev mode False
11:30:24,267 root INFO Starting session 8e46...
11:30:24,896 root INFO Getting refresh token...
11:30:25,312 root ERROR Failed to initialize Spark Lighter variables. HTTPSConnectionPool(host='wabi-north-europe-k-primary-redirect.analysis.windows.net', port=443): Max retries exceeded with url: /metadata/v201606/generatemwctoken (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))
11:30:25,312 root INFO Registering Spark Lighter magics for IPython...

1 ACCEPTED SOLUTION

HI @SnackSeeker91,

It seems mention that you are tried to use this data in multiple data regions. For this scenario, you can take a look the following link about multi-geo support of fabric capacity:

Multi-Geo support for Fabric - Microsoft Fabric | Microsoft Learn

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider accept as solution to help other members find it more quickly.

View solution in original post

5 REPLIES 5
SnackSeeker91
Regular Visitor

Hello @v-shex-msft, thank you very much for your reply. I have set the global proxy with setx for HTTP_PROXY, HTTPS_PROXY, and NO_PROXY. The connection and session are now working, but there's another problem with authentication. I can't read the tables in my OneLake.

 

Here is the log from the SparkLighter after session is connected.

Caused by: com.microsoft.aad.msal4j.MsalInteractionRequiredException: AADSTS50076: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access '00000009-....'. Trace ID: 6f682a... Correlation ID: 7a7d9c... Timestamp: 2024-10-28 12:22:25Z
at com.microsoft.aad.msal4j.MsalServiceExceptionFactory.fromHttpResponse(MsalServiceExceptionFactory.java:39) ~[msal4j-1.10.1.jar:1.10.1]
at com.microsoft.aad.msal4j.TokenRequestExecutor.createAuthenticationResultFromOauthHttpResponse(TokenRequestExecutor.java:96) ~[msal4j-1.10.1.jar:1.10.1]
at com.microsoft.aad.msal4j.TokenRequestExecutor.executeTokenRequest(TokenRequestExecutor.java:37) ~[msal4j-1.10.1.jar:1.10.1]
at com.microsoft.aad.msal4j.AbstractClientApplicationBase.acquireTokenCommon(AbstractClientApplicationBase.java:128) ~[msal4j-1.10.1.jar:1.10.1]
at com.microsoft.aad.msal4j.AcquireTokenByAuthorizationGrantSupplier.execute(AcquireTokenByAuthorizationGrantSupplier.java:63) ~[msal4j-1.10.1.jar:1.10.1]
at com.microsoft.aad.msal4j.AuthenticationResultSupplier.get(AuthenticationResultSupplier.java:59) ~[msal4j-1.10.1.jar:1.10.1]
at com.microsoft.aad.msal4j.AuthenticationResultSupplier.get(AuthenticationResultSupplier.java:17) ~[msal4j-1.10.1.jar:1.10.1]
at java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source) ~[?:1.8.0_311]
at java.util.concurrent.CompletableFuture$AsyncSupply.exec(Unknown Source) ~[?:1.8.0_311]
at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) ~[?:1.8.0_311]
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(Unknown Source) ~[?:1.8.0_311]
at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) ~[?:1.8.0_311]
at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) ~[?:1.8.0_311]

 

and notebook exception:

 

SnackSeeker91_0-1730191140311.png

(Querry and lakehouse are in the same Worspaces)

 

Thanks in advance for any help. Best regards.

HI @SnackSeeker91,

It seems mention that you are tried to use this data in multiple data regions. For this scenario, you can take a look the following link about multi-geo support of fabric capacity:

Multi-Geo support for Fabric - Microsoft Fabric | Microsoft Learn

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider accept as solution to help other members find it more quickly.

Hi @v-shex-msft ,

You are right we have different locations for Datalke and Azure so we need to migrate to the same location. After that it should work without hitting MSAL. Thank you very much for you help.

 

Best regards.

v-shex-msft
Community Support
Community Support

Hi @SnackSeeker91,

Have you tied to enable global proxy on your system instead of specific app? They should simple to use than manually setting program level proxy.
BTW, I check the official document but not found some of them mention about enable proxy in the vscode extension.

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider accept as solution to help other members find it more quickly.

Hello @v-shex-msft, thank you very much for your reply. I have set the global proxy with setx for HTTP_PROXY, HTTPS_PROXY, and NO_PROXY. The connection and session are now working, but there's another problem with authentication. I can't read the tables in my OneLake.

 

Here is a log from the SparkLighter:

SnackSeeker91_0-1730191696793.png

 

and here is the error from the jupyter notebook:

SnackSeeker91_1-1730191764347.png

(Querry and lakehouse are in the same workspace)

 

Thanks in advance for any help. Best regards.

 

Helpful resources

Announcements
November Carousel

Fabric Community Update - November 2024

Find out what's new and trending in the Fabric Community.

Live Sessions with Fabric DB

Be one of the first to start using Fabric Databases

Starting December 3, join live sessions with database experts and the Fabric product team to learn just how easy it is to get started.

November Update

Fabric Monthly Update - November 2024

Check out the November 2024 Fabric update to learn about new features.

Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Early Bird pricing ends December 9th.