Why is %29 in a URL Encode to ) During a Request?

[Suraj_Ncircle]

Hi Everyone,

I encountered an issue while working with a direct download URL in Power Query using the Web.Contents function. The URL contains an encoded value %29 (which represents a closing parenthesis ) in URL encoding). However, when the request is sent, the %29 is automatically converted to ) as observed in Fiddler.

Example:

URL I Want to Hit:
https://cdn.Domain.com/dXJuOmFkc2sud2lwcHJvZDpmcy5maWxlO_dmVyc2lvbj0x/output/Resource/3D%20View/2_%2...%29%20S-A_%20%EA%B5%AC%EC%A1%B0%2014159939/2_%20%EC%84%B1%EB%8A%A5%20%EA%B5%AC%EB%B6%84%29%20S-A_%20%EA%B5%AC%EC%A1%B0?{cookies Appended}

What I See in Fiddler:
https://cdn.Domain.com/dXJuOmFkc2sud2lwcHJvZDpmcy5maWxlO_dmVyc2lvbj0x/output/Resource/3D%20View/2_%2...)%20S-A_%20%EA%B5%AC%EC%A1%B0%2014159939/2_%20%EC%84%B1%EB%8A%A5%20%EA%B5%AC%EB%B6%84)%20S-A_%20%EA%B5%AC%EC%A1%B0?{cookies Appended}

It appears that %29 is being decoded to ) before the request is sent. This behavior causes 403 issues because the server expects the %29 to remain encoded.

My Questions:

1. Why is the %29 automatically decoded to ) in the request?
2. Is there a way to ensure %29 remains encoded in the request when using Power Query?
3. Are there any workarounds to handle this issue if the server strictly expects %29 instead of )?

Any insights or solutions would be greatly appreciated!

Thanks!

[v-csrikanth]

Hi @Suraj_Ncircle 
Thank you @BeaBF for your valuable insights and helpful suggestions! Your contribution is greatly appreciated.
Did you got chance to try the proposed solutions?
If you are still facing any challenges, kindly provide further details, and we will be happy to assist you.

Thanks,
Cheri Srikanth

[v-csrikanth]

Hi @Suraj_Ncircle 
I just wanted to check whether the solution provided has worked for you.
If yes, please accept as solution to help others benefit. If not, feel free to reach out.
Thank you.

[v-csrikanth]

Hi @Suraj_Ncircle 
Could you please review the updated code below to see if it meets your requirements?
*************************************************
let
OriginalUrl = "https://cdn.Domain.com/...",
EncodedUrl = Binary.ToText(Text.ToBinary(OriginalUrl), BinaryEncoding.Base64),
QueryResponse = Web.Contents("https://yourproxyserver.com", [Query = [encodedUrl = EncodedUrl]])
in
QueryResponse
*************************************************

If the above information helps you, please give us a Kudos and marked the reply Accept as a Solution.

Thanks,
Cheri Srikanth

[Suraj_Ncircle]

Unfortunately, the suggested solution didn’t work for me. The request is still not returning the expected response. The encoding used by Web.Contents differs from what the server expects, and due to re-encoding, some parts of the URL do not match the original one. I would appreciate any further guidance or alternative approaches to resolve this issue.

Additionally, why does Web.Contents re-encode an already encoded URL?

[v-csrikanth · Accepted solution]

Hi @Suraj_Ncircle 
While encoding the entire URL using Uri.EscapeDataString in PQ is problematic because Web.Contents expects a valid absolute URL and may not properly handle fully encoded URLs.
However, we can work around this by constructing the request in a way that ensures encoding remains intact.
Manually encoding the entire URL:
***************************************************************
let
BaseUrl = "https://cdn.Domain.com/dXJuOmFkc2sud2lwcHJvZDpmcy5maWxlO_dmVyc2lvbj0x/output/Resource/3D%20View/2_%2...",
Cookies = "your_cookie_data",
EncodedUrl = Uri.EscapeDataString(BaseUrl), // Encode the entire URL
QueryResponse = Web.Contents(EncodedUrl & Cookies, [Headers=[#"Accept"="application/json"]])
in
QueryResponse
***************************************************************
After giving a try if still the issue persists. Could you please follow below link to raise a support ticket
https://learn.microsoft.com/en-us/power-bi/support/create-support-ticket 

If the above information helps you, please give us a Kudos and marked the reply Accept as a Solution.

Thanks,
Cheri Srikanth

[v-csrikanth]

Hi @Suraj_Ncircle 
We haven't heard from you since last response and just wanted to check whether the solution provided has worked for you. If yes, please accept as solution to help others benefit. If not, feel free to reach out.
Thank you.

[Suraj_Ncircle]

Thank you for following up. I tried the solution provided by @BeaBF , but unfortunately, it didn’t resolve the issue. It seems that Web.Contents is unable to interpret %29 correctly. Specifically, we receive the encoded value %29 for 분) from the client server, but when the URL is processed, Web.Contents appears to interpret %29 as a closing parenthesis ).

I would appreciate any further guidance or suggestions to address this issue. Thank you for your understanding.

[BeaBF]

@Suraj_Ncircle Try with these two solutions:

1. let
BaseUrl = "https://cdn.Domain.com/dXJuOmFkc2sud2lwcHJvZDpmcy5maWxlO_dmVyc2lvbj0x/output/Resource/3D%20View/2_%2...",
Source = Web.Contents(BaseUrl)
in
Source

2. let
BaseUrl = "https://cdn.Domain.com/dXJuOmFkc2sud2lwcHJvZDpmcy5maWxlO_dmVyc2lvbj0x/output/Resource/3D%20View/2_%2...",
Headers = [#"User-Agent"="Mozilla/5.0"],
Source = Web.Contents(BaseUrl, [Headers=Headers])
in
Source

let me know.

BBF

[v-csrikanth]

Hi @Suraj_Ncircle 

I wanted to check if you had a chance to try the suggested solution.
Please let us know if you need any further assistance.

Looking forward to your update!

If the above information helps you, please give us a Kudos and marked the reply as a solution.

Thanks,
Cheri Srikanth

[v-csrikanth]

Hi @Suraj_Ncircle 
Thank you for being part of the Microsoft Fabric Community.

As highlighted by @BeaBF, the proposed approach appears to effectively address your requirements. Could you please confirm if your issue has been resolved.

If you are still facing any challenges, kindly provide further details, and we will be happy to assist you.

If the above information helps you, please give us a Kudos and marked the reply as a solution.

Thanks,
Cheri Srikanth

[BeaBF]

@Suraj_Ncircle Hi! 

This issue arises because the Web.Contents function in Power Query may automatically normalize or decode parts of the URL before sending the request. Specifically, %29 (representing a closing parenthesis) is being decoded to ) during this process.

You can try using Uri.EscapeDataString to explicitly encode the URL again or manipulate the raw HTTP request if possible. Here's an example workaround in Power Query:
let
BaseUrl = "https://cdn.Domain.com/dXJuOmFkc2sud2lwcHJvZDpmcy5maWxlO_dmVyc2lvbj0x/output/Resource/3D%20View/2_%2...?",
Cookies = "your_cookie_data",
EncodedUrl = Text.Replace(BaseUrl, "%29", "%2529"), // Double encode %29 to %2529
QueryResponse = Web.Contents(EncodedUrl & Cookies)
in
QueryResponse

BBF