Gateway needed for web data ?

[cnschulz]

Hi, 

I have created a simple test report that gets data from a publicly available website. It simply gets the current weather from a provider and shows a graph over time. 

When the report is loaded to app.pwerbi.com, the dataset created cannot be refreshed as it stated I need a data gateway for this. I was of the belief that for direct connect to the web I did not need a gateway.

Could someone confirm this? Is there a way for me to source data from the web without a gateway?

Cheers

[GilbertQ]

Hi there

It depends on how the underlying M Code is being used.

I cannot recall off the top of my head, but I am certain there are different ways.

[GilbertQ]

Hi there

I got this from a good friend of mine

Web.page function needs gateway. If you use web.contents you won't need it.

So just check the underlying M Code.

[cnschulz]

Thanks for your quick reply.

I am using the "out of the box" web data source. When I add my URL the following code is added as a source.

= Web.Page(Web.Contents("http://www.bom.gov.au/tas/observations/tasall.shtml"))

If I remove the "Web.Page()" and leave the Web.Contents, I am shown an icon of the website and my only option is to click to expands the data, which regenerates the Web.Page() wrapper.

When I attempt to set up refresh on the generated dataset, i am prompted to use a gateway. 😞

Any ideas?

[cnschulz]

Hi, 

The very first source line of that linked page is identical to mine 

Source = Web.Page(Web.Contents("http://..." 

 Therefore, it should require a web gateway as well. 

Has anyone processed web (http) data in a report on the powerbi service without a gateway? Is so please detail the steps.

[GilbertQ]

Hi there

As explained above it can only be done with a Web.Contents, if there is a Web.Page it will require a gateway.

[kirvis]

Hi,

Don't know if this is still actual, but I ran into the same issues.

I found a workaround, which I describe here: https://community.powerbi.com/t5/Power-Query/WORKAROUND-How-to-use-scheduled-refresh-of-a-web-page-w....

Hope that helps!

Kirvis

[luisrh]

You are screen scraping.  Is there a web service end point that returns JSON or CSV for this site?   You can hand code M to do the expansion.   Here is a post that can help you.      https://community.powerbi.com/t5/Desktop/Web-scraping-advice/td-p/330414

[GilbertQ]

Not off the top of my head, but I am sure that there is a way to get this done.

[cnschulz]

Anyone? I must be missing something basic here.

---

@GilbertQ wrote:
Not off the top of my head, but I am sure that there is a way to get this done.

---