Forum Discussion
push data into Power BI Dataset from asp.net web api
Hi,
I am getting my data for power bi report from asp.net web api as json. As i am fetching 50-60k records, it takes time to get all data and if there is network traffic, request timeout also happens sometimes.
So my question is:
Is there a way to populate dataset as a stream means instead of getting all the data at once. can i fetch some data pushing it to dataset and so on? It is not a live streaming scenario.
Any pointers will be very appreciated.
Thanks.
Hi nasirali,
If you push data to streaming api, it should be streaming dataset.(streaming dataset not cache data, it can only use with specific tiles)
If you mean publish data to rest api and create dataset who stored in power bi service. It should be a normal dataset.
I'm not so sure for timeout issue which you mentioned, but I think you can write a loop function to split these data to suitable size/interval and push them.
For power bi rest api limitation, please refer to below link:
To POST Rows
- 75 max columns
- 75 max tables
- 10,000 max rows per single POST rows request
- 1,000,000 rows added per hour per dataset
- 5 max pending POST rows requests per dataset
- 120 POST rows requests per minute per dataset
- If table has 250,000 or more rows, 120 POST rows requests per hour per dataset
- 200,000 max rows stored per table in FIFO dataset
- 5,000,000 max rows stored per table in ‘none retention policy’ dataset
- 4,000 characters per value for string column in POST rows operation
Regards,
Xiaoxin Sheng
4 Replies
Hi nasirali,
What type of power bi you used ? Desktop or service? If you mean desktop, current it not support get streaming data.
If you mean power bi service, you can simply setting an application to push your data to streaming dateset api.
Real-time streaming in Power BI
Push Data into Power BI Streaming Datasets with C#
But current power bi not support to use steaming dateseting on normal visuals.
Regards,
Xiaoxin Sheng
Hi Anonymous
Thank you for reply and links. I am using power bi desktop for development of reports then publish to service for embedding in application. I have a report which is getting data from webapi as json and data is being updated with refresh schedule. Everything is working fine.
Only problem is, web api has a request timeout. And if data is large enough it will timeout, So i was thinking if there is a way to push data as i read it. It will be efficient and i will not have this problem. Something like in the second the link you provided. Will it work with normal dataset too or just streaming dataset?
Regards,
nasir ali
Hi nasirali,
If you push data to streaming api, it should be streaming dataset.(streaming dataset not cache data, it can only use with specific tiles)
If you mean publish data to rest api and create dataset who stored in power bi service. It should be a normal dataset.
I'm not so sure for timeout issue which you mentioned, but I think you can write a loop function to split these data to suitable size/interval and push them.
For power bi rest api limitation, please refer to below link:
To POST Rows
- 75 max columns
- 75 max tables
- 10,000 max rows per single POST rows request
- 1,000,000 rows added per hour per dataset
- 5 max pending POST rows requests per dataset
- 120 POST rows requests per minute per dataset
- If table has 250,000 or more rows, 120 POST rows requests per hour per dataset
- 200,000 max rows stored per table in FIFO dataset
- 5,000,000 max rows stored per table in ‘none retention policy’ dataset
- 4,000 characters per value for string column in POST rows operation
Regards,
Xiaoxin Sheng
