Excel Get Data from Web Timeout

[ExcelUser_921]

I posted the following message in the Microsoft Community forum and got routed here because of the Power-Query related problem. It was suggested that the connection timeout might be related to the website's anti-crawling mechanism. The support specialist helping me also failed to retrieve data from the website using the same operation. Any help would be greatly appreciated!

I have to update the future energy pricing everyday for oil and gas. Currently I'm just copying and pasting from the webpage, but I'd like to use the Get Data from Web function in Excel to connect to the CME group and pull the entire data table into excel for that specific day.

Future Oil Price
I am interested only in the "Month" and "Settle" columns.

Future Gas Price

I am interested only in the "Month" and "Settle" columns.

Workflow

Get Data > From Web > paste URL

Excel takes about 10 minutes to connect, but ultimately times out.

I'd appreciate some help in finding a way to extract this data automatically.
Thank you for your help!

[PhilipTreacy]

@ExcelUser_921 

I don't think that video will help as it looks like it's about getting data from 'regular' websites i.e. those that don't implement anti-scraping technology or use JavaScript to load pages

Even using a scraping proxy like ScrapingBee doesn't work.  Companies like this live off the data they provide so they don't want people scraping their web pages.

If you check their terms it's even explicitly stated that scraping is forbidden (under Prohibited Uses) Market Data Explanation and Disclaimer

use of any Web browsers (other than generally available third-party browsers), engines, scripts, software, spiders, robots, avatars, agents, tools or other devices or mechanisms (such as crawlers, browser plug-ins and add-ons, or other technology) to navigate, access, copy in bulk, retrieve, harvest, index, search or analyze any portion of the Website is strictly prohibited.

Regards

Phil

[Omid_Motamedise]

See this video it is exactly whay tou want

https://www.youtube.com/watch?v=NdUZx_yyEqY&pp=ygUIZnJvbSB3ZWI%3D