Streaming Dataset from a JSON API

[olimilo]

Hi everyone,

So we are pulling data from a websource using a JSON API. However, the data that we get from the JSON API is not laid out in a flat file-like structure but has columns that look like the one below:

data

{List}

1/1/2020

Passed

So basically, I have to transform the data first to get it to something more usable. My question is, the PBI streaming dataset looks like it requires to have your file in a flat file-like structure. Since my data isn't flat file-like in the first place, how can I achieve a streaming dataset with my data?

[amitchandak]

@olimilo , check if this can help

https://bpcs.com/blog/power-bi-streaming-data-sets-the-good-the-great-and-the-gotchas

[Greg_Deckler]

@olimilo - Use Azure Streaming Analytics to transform your data prior to getting to Power BI.

[olimilo]

Hi @Greg_Deckler @amitchandak 

I tried refreshing the data using the dataset tab in the PBI service and got this error: Query contains unsupported function. Function name: Web.Contents.

Which is funny because when I refreshed the data in PBI desktop, the refresh went without a hitch. Will this have any bearing on us trying to achieve a realtime connection to our data? This is the connector that I used:

= Json.Document(Web.Contents(JSONSourceURL, [Timeout=#duration(0, 0, 30, 0), Headers=[#"User-Agent"="Power BI", Accept="application/jsonapi", #"Content-Type"="application/json", Authorization="..."]]))

Edit: I saw on the reference page below that JSON data sources does not support DirectQuery/Live connection - does that mean that we're out of options in this case? Or is it not possible for us to schedule data refreshes?

https://docs.microsoft.com/en-us/power-bi/connect-data/power-bi-data-sources