Forum Discussion

jmillsjmills's avatar
jmillsjmills
Helper III
3 years ago

Scraping Web Data

Hello - I'm looking to web scrape data from the following URL.

Football Betting & Football Odds ⇒ William Hill™

 

You'll note from my screenshot that I need to click on "Stats" to open up the data on the page so it's visible in the browser, but that this doesn't change the URL. Is there any way I could open the URL with a #element referenced directly, so the page opens as though this element has already been clicked?

 

Alternatively, my original way around this was to look for the call within the network when the pages loads. It looks as though the source containing the data I need is within here:

https://w.sports.williamhill.com/fragments/eventEntity/en-gb/football/OB_EV23597695/6

 

This worked initially but now appears to boot me out - possibly to do with something called Cloudfront preventing me? Is this something to actively block web-scraping? Or do I just need something else in my request there? Thanks

 

Either solution would be great please - I'd only need the one. Either the way of loading the page as though "stats" has been clicked, or accessing this seemingly inaccessible API somehow. Thanks so much

 

 

2 Replies