Forum Discussion

jmillsjmills's avatar
jmillsjmills
Helper III
3 years ago

Another Web Scraping Query

I'm looking at pulling in data from this website:

https://www.betfred.com/sports/event/22986947.2

 

And it looks as though the embedded call to the data is via this link, which (as per the screenshot) you can see parses nicely in the "Preview" tab when you inspect into the code...

https://www.betfred.com/services/SportsBook/event?language=uk&type=event&eventid=22385490.2&dataflags=14&datasize=8&cachebust=1668982826349

 

However, when I'm pulling it back, Power Query is determined to treat it as HTML, which it obviously fails to parse. My best attempt at forcing it to treat it as JSON is:

 

= Json.Document(Web.Contents("https://www.betfred.com/services/SportsBook/event?language=uk&type=event&eventid=22385490.2&dataflags=14&datasize=8&cachebust=1668982826349"))

 

I feel like it's something to do with needing to force other headers into the query. But PLEASE PLEASE HELP ME.

 

Thanks so much!

 

 

 

Need Help

17 Replies

  • v-yueyunzh-msft's avatar
    v-yueyunzh-msft
    Community Support
    3 years ago

    Hi , jmillsjmills 

    According to your description, you want your Http request to return a string in JSON format. Right?

    First , you can put your request and request parameters into the "Postman" tool for debugging and see if it returns JSON format.
    Second, you can try adding in the "Headers"

    Accept=application/json 

     

    For how to add "Headers" in Web.Contents , you can refer to :

    Web.Contents - PowerQuery M | Microsoft Learn

     

     

    Thank you for your time and sharing, and thank you for your support and understanding of PowerBI! 

     

    Best Regards,

    Aniya Zhang

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly

     

    • jmillsjmills's avatar
      jmillsjmills
      Helper III
      3 years ago

      Thanks so much for coming back Aniya!

       

      Sorry, I'm not quite following. Are you able to test on your side whether you can pull this data in? The link you've provided exactly the sort of thing I've been playing with but I simply cannot work out how to pull the data.

       

      If you are able to kindly provide what the Power Query code would need to be exactly in this instance, I would be so very grateful!

       

      Thank you so SO much!

      • v-yueyunzh-msft's avatar
        v-yueyunzh-msft
        Community Support
        3 years ago

        Hi , jmillsjmills 

        This url is your internal , i cannot test in my workaround, but you can try to use this and replace the url for yours to test .

        let
    url = "https://www.betfred.com/services/SportsBook/event?language=uk&type=event&eventid=22385490.2&dataflag...[your url should replace here]",
    headers = [#"Accept"="application/json"],
    response = Web.Contents(
        url,
        [
            Headers = headers
        ]
    ),
    jsonResponse = Json.Document(response)
in
    jsonResponse

         

        Thank you for your time and sharing, and thank you for your support and understanding of PowerBI! 

         

        Best Regards,

        Aniya Zhang

        If this post helps, then please consider Accept it as the solution to help the other members find it more quickly