Error with Web.Contents

[jpessoa8]

Hi everyone,

I've created a query to the Microsoft events page to track the next DIAD and PRIAD in Portuguese (all covered by the company BI4ALL in Portugal) to alert newcomers on the next event for them to assist this sessions which is a nice, fast and cheap way for them to learn about these tools.

Since I didn't want to use a gateway for this, I've been using the Web.Contents to obtain the information from the website and everything was working fine until 4 days ago (2nd of March 2023) the Dataset started failing the refresh with a "(500): Internal Server Error". When I try and do the same on the Desktop I get the same error.

I've tried using Web.BrowserContents, which requires a gateway on the Service, and it works on the Desktop but still doesn't work on the Service (not even when trying to create via DataFlow).

Do you have any explanation for this?

I leave the query that I am using.

Thanks in advance.

let
    Source = Html.Table(Web.Contents("https://events.microsoft.com/pt-br/allevents?search=bi4all"), 
        {
        {"Column1", ".c-heading-6"}, 
        {"Column2", ".title-date"}, 
        {"Local", ".digital"}, 
        {"Link", "a[data-clipboard-text^=""http""]", each [Attributes][#"data-clipboard-text"]}
        }
    , [RowSelector=".c-card"]),
    #"Inserted Formação" = Table.AddColumn(Source, "Formação", each Text.BeforeDelimiter([Column1], " -"), type text),
    #"Inserted Date" = Table.AddColumn(#"Inserted Formação", "Date", each Text.BeforeDelimiter([Column2], " "), type text),
    #"Removed Columns" = Table.RemoveColumns(#"Inserted Date",{"Column1", "Column2"}),
    #"Changed Type" = Table.TransformColumnTypes(#"Removed Columns",{{"Link", type text}})
in
    #"Changed Type"

[ichavarria · Accepted solution]

The HTTP 500 Internal Server Error message indicates that the server encountered an unexpected condition that prevented it from fulfilling the request made by the client (your Power BI query).

Unfortunately, there could be many reasons why this error is occurring. It could be a temporary issue with the website, a change in the website's structure that broke your query, or even a security feature implemented by the website to prevent scraping.

If the issue is temporary, you can try refreshing your query again later to see if it resolves itself. If not, you may need to update your query to accommodate any changes made to the website's structure. It is also worth checking with the website's administrator to see if there have been any changes made that could affect your query.

As for the issue with using Web.BrowserContents, it is possible that your organization's firewall is blocking the connection to the website, or that the website is blocking automated requests made by Power BI. If possible, you can try using a personal device with a different internet connection to see if the issue persists. If it does, then it is likely an issue with the website's security measures. In this case, you may need to contact the website administrator to see if they can provide a workaround for accessing their data through automated means.

Best regards, 

Isaac Chavarria 


If this post helps, then please consider Accept it as the solution to help the other members find it more quickly