Forum Discussion

jmillsjmills's avatar
jmillsjmills
Helper III
3 years ago

Another Web Scraping Query

I'm looking at pulling in data from this website: https://www.betfred.com/sports/event/22986947.2   And it looks as though the embedded call to the data is via this link, which (as per the screens...
Need Help
v-yueyunzh-msft's avatar
v-yueyunzh-msft
Community Support
3 years ago

Hi , jmillsjmills 

For the url , I need to determine if you're using the correct URL.

For URLs, you need to use your URL in "F12" instead of your URL at the top of the browser.

As prompted, your URL returns an HTML format.

You need to make sure that the URL used is the Request URL you showed in the screenshot.

Second, you can also check "F12" to see if there are other "headers" that need to be added.

 

Thank you for your time and sharing, and thank you for your support and understanding of PowerBI! 

 

Best Regards,

Aniya Zhang

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly

 

  • jmillsjmills's avatar
    jmillsjmills
    Helper III
    3 years ago

    That's exactly what I am doing yes. When I'm copying and pasting the URL onto here, it seems to be shortening it. But if you click through to the link, that's the full URL I am using. But for whatever reason I'm still getting this same error as shown in yellow on the previous screenshot. I know I am being silly but please spell out how I can get Power Query to process this information as JSON rather than HTML. Do you have this exact URL working on your side please? It clearly provides functional JSON given this is how it previews in the "Inspect" feature on a web browser. But Power Query is determined to treat it as HTML... Thank you so so much for your patience with my slow understanding!

    • v-yueyunzh-msft's avatar
      v-yueyunzh-msft
      Community Support
      3 years ago

      Hi , jmillsjmills 

      For the test , it returns the json for this url:

      let
    url = "https://www.bing.com/videos/vdasync?w=298&mid=A5DDD429C801C67FCF7CA5DDD429C801C67FCF7C",
    headers = [#"Accept"="application/json"],
    response = Web.Contents(
        url,
        [
            Headers = headers
        ]
    ),
    jsonResponse = Json.Document(response)
in
    jsonResponse

      This is return json type , i think you need to check your url and the parameters in your code . For this , you can try to use "postman" to send the request to test if the url return the json type.

      Postman API Platform

       

      Thank you for your time and sharing, and thank you for your support and understanding of PowerBI! 

       

      Best Regards,

      Aniya Zhang

      If this post helps, then please consider Accept it as the solution to help the other members find it more quickly

       

      • jmillsjmills's avatar
        jmillsjmills
        Helper III
        3 years ago

        Your above code is what I've already tried - and it results in the same error:

        DataSource.Error: The downloaded data is HTML, which isn't the expected type. The URL may be wrong or you might not have provided the right credentials to the server.

         

        Is there no way you'd be able to please test this specific URL I am trying to use on your side? Thanks so much again