topic Re: Batch add rows via Logic App Connector? in Developer

Batch add rows via Logic App Connector?

ThreeTen22 — Tue, 10 Apr 2018 08:36:43 GMT

Hey Guys. I currently am trying to create a logic app that pushes ~50 items every 10 seconds into PowerBi via the logic app connector. Due to the connectors limitations ill constantly hit the cap very quickly. Looking at the Connector information there seems to be a "code only" optional parameter called "Payload" that I can use rather than the single row form that automatically displays.

However, in logic app's code designer I have tried setting Payload as an object variable, so its output would match your REST API call:

"Payload" : 
{
  "rows": 
            [ 
                {"ProductID":1,"Name":"Adjustable Race","Category":"Components","IsCompete":true,"ManufacturedOn":"07/30/2014"}, 
                {"ProductID":2,"Name":"LL Crankarm","Category":"Components","IsCompete":true,"ManufacturedOn":"07/30/2014"}, 
                {"ProductID":3,"Name":"HL Mountain Frame - Silver","Category":"Bikes","IsCompete":true,"ManufacturedOn":"07/30/2014"} 
            ] 
}

As well as modify the variable to that of an array, so the connector's output would look like this:

"Payload": [ 
{"ProductID":1,"Name":"Adjustable Race","Category":"Components","IsCompete":true,"ManufacturedOn":"07/30/2014"}, 
{"ProductID":2,"Name":"LL Crankarm","Category":"Components","IsCompete":true,"ManufacturedOn":"07/30/2014"}, 
{"ProductID":3,"Name":"HL Mountain Frame - Silver","Category":"Bikes","IsCompete":true,"ManufacturedOn":"07/30/2014"} 
]

However both variations do not work. As this functionality has been in Power BI's REST API for awhile, on top of the fact that the url string in the logic app connector itself goes to the same url as your REST API I can't imagine this cannot be done?

Re: Batch add rows via Logic App Connector?

v-micsh-msft — Wed, 11 Apr 2018 10:40:03 GMT

Is your DataSet a real time streaming DataSet?

The Dataset for the Power BI connector in Logic Apps, PowerApps and Microsoft Flow currently only supprts RealTime Streaming Dataset.

If not, then the Add rows to DataSet will not work.

Regards,

Michael

Re: Batch add rows via Logic App Connector?

ThreeTen22 — Fri, 13 Apr 2018 00:35:40 GMT

To clarify, are you saying that you can only submit an array of objects to the connector if the dataset is a "RealTime Streaming Dataset"? (it seems in this case it is a PushStreaming dataset with historical data turned off? / When looking at the dataset online it's API Access field is "Streaming")

Assuming that is the case I gave it a shot:

The Dataset:

There is a single column, "Count", that accepts a Number.

Giving Payload a single object works fine:

but give it an array and you get the error:

This is the same error I got when I tried using PushStreaming, Push & Multi-Table Push datasets. They all work however if I only provide a single object.

I ran into an old powebi blog article which showed the connector accepting an array of objects:

I don't know if the devs reworked the connector somehow?

Re: Batch add rows via Logic App Connector?

v-micsh-msft — Fri, 13 Apr 2018 05:24:18 GMT

Thanks for the update.

The connector under Microsoft Flow seems to be updated.

The available DataSet listed are all streaming dataset, and the fields is generated automatically. So I am afraid the payload may not work at the current time.

You may consider add an apply to each loop in Logic Apps:

https://docs.microsoft.com/en-us/azure/logic-apps/logic-apps-control-flow-loops

Regards,

Michael

Re: Batch add rows via Logic App Connector?

ThreeTen22 — Wed, 18 Apr 2018 01:22:41 GMT

considering each call in a while/foreach loop counts as a Connector call, the maximum rate of rows I can push per second is ~1.6 rows per second, (100 calls / 60s) , which is laughably unscalable.. I guess I will have to code this solution after all.

(I would also suggest renaming the connector action to "add_row_to_dataset" as you cannot add more than one at a time).

Thanks for your time