Schedule Refresh is failing - Data Source Credentials for the HTTP request response

[KR300]

Hi @Team,

My team is publishing release documents in the Cloud_web for each release after a certain data ( no fixed Dates ).

By using a power query custome function, i am getting a response code from an http request ( i gave an http URL  in custome function ) in Power bi desktop to show to the USER/LEADERSHIP THAT  link is available or not. like below

After writing the custome function; I am getting below info

Status: 404/200

Message: Link is not Available/Link is Available

In PBI Desktop using DAX I am giving Dynamic fill color to the Card value based on the Message to the URL ( Dynamic URL gave bu using button placing top of the Card visual ). So that the User/Leadership get to know that the link info available or not.

SFP.R.12.1 (  Link is Available )

SFP.R.12.2  ( Link is not Available )

Moreover, I have other datasourece ( Sharepoint excel + Other Cloud sources - I have given those credentials in service already )

Data Source Credentials:( DSC )

While giving credentials for the http request URL ( Let' say SFP.R.12.2  ( Link is not Available ) ). 

Do i need to give credentials for that URL as well.

I gave the DSC but i am getting error ( Beacuse the document is not published for the SFP.R.12.2 that is why i am getting 404 in thebelow message ) -- Will it impact the Schedule Refresh(SC)? & If I skip the test connection, will it give SC success.

( Failed to update data source credentials: Web.Contents failed to get contents from 'https://amedeloitte.sharepoint.com/sites/xxxxxxxxxxxxx/_api/web/getfilebyserverrelativeurl('/sites/D...' (404): Not Found )

[v-menakakota · Accepted solution]

Hi @KR300 

Thank you for reaching out to us on the Microsoft Fabric Community Forum.

We really apologize for the inconvenience, after reviewing the issue of schedule refresh is failing data source credentials for the HTTP request response, here are few steps to may resolve the issue.

• Make sure that all data source credentials are properly configured and authenticated in the Power BI Service.
• Verify that the HTTP request URL is accessible and returns the expected response. If the document hasn’t been published, consider implementing a way to handle 404 errors gracefully in your custom function.
• Adjust the custom function to manage 404 errors without causing the entire refresh to fail. You can return a default value or log the error for further analysis.
• If possible, isolate the HTTP request URL query from the other data sources to identify the issue more easily and prevent it from impacting the overall refresh process.
• The team utilizes additional data sources, including SharePoint Excel and other cloud services, for which the credentials have already been configured in the service. When attempting to enter credentials for the HTTP request URL, they encounter an error because the document has not been published, resulting in a 404 error.

Here is the link to the documentation that might assist in resolving the issue.
https://learn.microsoft.com/en-us/power-bi/connect-data/refresh-scheduled-refresh

If this post was helpful, please give us Kudos and consider marking Accept as solution to assist other members in finding it more easily.

Regards,
Menaka.

[v-menakakota · Accepted solution]

Hi @KR300 

Thank you for reaching out to us on the Microsoft Fabric Community Forum.

We really apologize for the inconvenience, after reviewing the issue of schedule refresh is failing data source credentials for the HTTP request response, here are few steps to may resolve the issue.

• Make sure that all data source credentials are properly configured and authenticated in the Power BI Service.
• Verify that the HTTP request URL is accessible and returns the expected response. If the document hasn’t been published, consider implementing a way to handle 404 errors gracefully in your custom function.
• Adjust the custom function to manage 404 errors without causing the entire refresh to fail. You can return a default value or log the error for further analysis.
• If possible, isolate the HTTP request URL query from the other data sources to identify the issue more easily and prevent it from impacting the overall refresh process.
• The team utilizes additional data sources, including SharePoint Excel and other cloud services, for which the credentials have already been configured in the service. When attempting to enter credentials for the HTTP request URL, they encounter an error because the document has not been published, resulting in a 404 error.

Here is the link to the documentation that might assist in resolving the issue.
https://learn.microsoft.com/en-us/power-bi/connect-data/refresh-scheduled-refresh

If this post was helpful, please give us Kudos and consider marking Accept as solution to assist other members in finding it more easily.

Regards,
Menaka.

[KR300]

Thanks for the info . i will go through the link

Here is the my Power query for the HTTP Response:

//(url as text) as record =>
let
response = Web.Contents("https://amedeloitte.sharepoint.com/sites/DOL-c-US-SMPinfo/SitePages/External-Release-Notes-"& FixVersionName &".aspx", [ManualStatusHandling={404,400,500}]),
responseStatus = Value.Metadata(response)[Response.Status],
result =
if responseStatus = 200 then [Status = 200, Message = "Link is Available"]
else if responseStatus = 404 then [Status = 404, Message = "Link is not Available"]
else if responseStatus = 400 then [Status = 404, Message = "Link is not Available"]
else [Status = responseStatus, Message = "Other Error"]
in
result

Result:

Status: 404 / 200

Message: Link is not Available / Link is not Available

Based on the Info : I am showing  in the report using Card visual / button ( for the dynamic URL purpose I

 SFP.R.12.1    ( Red color ) + showing ( Link is not Available / Link is not Available ).

@v-menakakota @GilbertQ  can you pls help me how to customise the below query or we need to isolate that one . Because In PBI desktop, we are getting http request info after refreshing but in PBI service getting error if the document is not published yet causing entire schedule refresh failing.

[v-menakakota]

Hi @KR300 ,

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

[KR300]

I isolated the http request as a separate table + published + I did the Skip test Connection +

But i have one doubt.

Suppose if the document is available ( But we skipped the test connection in the PBI service ). Will it get latest data? into the reprot. 

Separating the HTTP request into a standalone table that refreshes independently ( What do you mean by this Pls explain ).

Using dataflows to manage this step separately from other sources. -- Can we solve this using Dataflows & how?

[v-menakakota]

Hi @KR300 ,

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

[v-menakakota]

Hi @KR300 ,

Skipping the test connection only impacts credential validation during setup and does not affect data retrieval during scheduled refreshes. Once the document becomes available, Power BI Service will successfully access it in the next scheduled refresh, fetch the latest data, and update the report.

The HTTP request is part of your main dataset refresh, causing the entire refresh to fail when the document is not available. Separating it into a standalone table means,creating a new table in Power Query dedicated to fetching the HTTP status of the document.Refreshing this table separately from the main dataset.
It can be done like below:

1. Create a separate Power Query table for the HTTP request response.
2. Load it into Power BI as a separate table ,do not merge it with the main dataset.
3. Schedule the refresh for this table independently, so it does not interfere with other datasets.

By doing this, even if the HTTP request fails , it won’t impact the rest of the report refresh.

Yes, using Dataflows can help manage this step separately. Here’s how:

1. Create a Dataflow in Power BI Service that only contains the HTTP request query.
2. Schedule the Dataflow to refresh separately
3. In Power BI Desktop, connect your main dataset to this Dataflow instead of making direct HTTP requests.
4. Since the Dataflow refreshes independently, if the HTTP request fails, it won’t affect the main dataset refresh.
5. The main dataset will simply fetch the latest successful result from the Dataflow without failing.

If this post was helpful, please give us Kudos and consider marking Accept as solution to assist other members in finding it more easily.
Regards,
Menaka.

[v-menakakota]

Hi @KR300 ,

I hope this information is helpful. Please let me know if you have any further questions or if you'd like to discuss this further. If this answers your question, please Accept it as a solution and give it a 'Kudos' so others can find it easily.

Thank you.

[v-menakakota]

Hi @KR300 ,

I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.

Thank you and Regards,
Menaka.

[v-menakakota · Accepted solution]

Hi @KR300 ,

We really apologies for the inconvenience.
Enable "Skip Test Connection" in Power BI Service. Since this HTTP request URL might not always be available, try skipping the test connection when setting up the credentials in Power BI Service:

• Navigate to Power BI Service and go to the Dataset settings.
• Find the HTTP data source.
• Click on Edit credentials.
• Enable the option to Skip test connection.
• Save the changes and try the scheduled refresh again.
  
  If the HTTP check is part of a larger dataset refresh, consider:
• Separating the HTTP request into a standalone table that refreshes independently.
• Using dataflows to manage this step separately from other sources.
  
  If this post was helpful, please give us Kudos and consider marking Accept as solution to assist other members in finding it more easily.
  
  Regards,
  Menaka.

[GilbertQ]

Hi @KR300 

And my understanding, if you get a 404 error, this could be managed so that it doesn't actually create an error when the semantic model is refreshing. So I think as long as you have that covered in your code, it should work as expected.