Forum Discussion

jmillsjmills's avatar
jmillsjmills
Icon for Helper III rankHelper III
5 years ago

Web Scrape: Worked before but now timing out...

Hello - can anyone find a way of scraping Bet365 without it timing out? https://www.bet365.com/#/AC/B1/C1/D8/E100842460/F3/I8/   I had a code that would work for this URL previously, but several mo...
Need Help
v-angzheng-msft's avatar
v-angzheng-msft
Icon for Community Support rankCommunity Support
5 years ago

Hi, jmillsjmills 

 

I seem to have found a workaround, try to change Web.BrowserContents to Web.Contents

I tried the following M code, PowerQuery returned me the correct Html information

let
    url="https://www.bet365.com/#/AC/B1/C1/D8/E100842460/F3/I8/",
    web=Text.FromBinary(Web.Contents(url))
in
    web

For references:

https://community.powerbi.com/t5/Desktop/Web-Scraping-Web-BrowserContents-Html-Table-Errors/m-p/1499433

 

Hope this helps.

 

Best Regards,
Community Support Team _ Zeon Zheng
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

  • jmillsjmills's avatar
    jmillsjmills
    Icon for Helper III rankHelper III
    5 years ago

    Thank you so much for your reply! That's very useful to know and I appreciate the workaround, given I had given up!!

     

    It looks like the HTML it is now pulling in is a little different to before. I was using CSS selectors to plot an HTML table that pulled in the odds (for example I was looking for .srb_ParticipantLabelWithTeam_Name, .srb_ParticipantLabelWithTeam_Team and .gl-MarketGroupButton_Text, all with a row selector set as .srb-ParticipantLabelWithTeam). You can see these CSS selectors if you inspect element on the odds in the website directly.

     

    However the HTML it's pulling in for me (and not sure it's the same for you) appears to be one large booting javascript function of some sort? Not the best at diagnosing these things but there certainly doesn't seem to be the full page content in the same way as before. Is this the case for you? Do you have any more ideas?

     

    Thank you so much! The link may now have expired but this URL is a new match:

    https://www.bet365.com/#/AC/B1/C1/D8/E100693610/F3/I8/

     

    Really appreciate all your efforts!