Ingest data via a webhook

[Anonymous]

In order to access automatic reporting from a service we use, I'm required to create a webhook URL, where they will send data to (as opposed to retrieving data from an API). I was wondering if this system could be managed in some way using Power Query within Power BI? I.e. be able to create a URL which enables the data to be sent straight to the Power BI dataset in question. I can't see any obvious way to do it, so I'm assuming it isn't possible but if anyone has any ideas, they would be greatly received.

[Anonymous]

It is my understanding that a webhook is a push api that required a client end point, as you descirbed. It is my understanding that you can create this end point in the power bi serivce. It is called a "streaming dataset" in powerbi terminology. https://docs.microsoft.com/en-us/power-bi/connect-data/service-real-time-streaming 

[mahoneypat]

While it is possible to set up a flow with a URL that can receive data you can then send to a Power BI dataset, I think it would be easier to have them email you the data in a consistent format (Excel, CSV, JSON, XML), and have a flow that is triggered from that email and then moves to file to SharePoint/OneDrive, and trigger refresh of your Power BI dataset that uses that SharePoint/OneDrive as the source.

Pat

[Anonymous]

Checked with them and don't think it's going to be possible @mahoneypat. What would be the best way to create a URL that is capable of storing data in that case? Use a paid solution or try and code it up myself in python, say, which I have a reasonable knowledge and understanding of but I've never done data extraction this way around (normally use APIs rather than webhooks).

[mahoneypat]

You can adapt the approach in this article (others out there too) to send data to a URL that triggers a Flow in which you can a save the data to SharePoint/OneDrive and then refresh the dataset that uses it as the source.

Chris Webb's BI Blog: Calling Microsoft Flow From Power Query And Power BI Chris Webb's BI Blog (cro...

Pat

[mahoneypat]

Try using the Web connector and pasting your url into the popup box, and see how that works.  Do you know what type of authentication your webhook expects?

Pat

[Anonymous]

Thanks @mahoneypat. Problem is, I actually need to create the URL in the first place. Is there any way I can do this through Power BI, maybe by creating an application, or do I have to do it another way? If the latter, what's the best method of doing it that is free (coding it using Python for example)? I've seen companies like Hevo that provide the ability to create a webhook URL where data can be sent and stored on their servers but they charge £200 a month, which I'm trying to avoid.