Checking retention policy for existing datasets

[taoyang]

After a dataset is created, is there a way to check the retention policy? I have created some datasets using REST API described in this blog article: https://powerbi.microsoft.com/en-us/blog/automatic-retention-policy-for-real-time-data/

But after the dataset is created, I can't find a way to check the retention policy.

Thanks

Tao

[SudhamaiMannam]

All of the Data is stored in a storage i.e. Azure Data Lake by Power BI Service which is not directly accesible.

If you create a streaming dataset it is only accessibile via api. You can not directly delete a row from the dataset, the only way will be delete all rows via the api. A solution for your purpose can be, delete the rows in Power BI Desktop in the Editor after you connect to your dataset. The default rention is 200k and is handeld Fifo see the Picture Below from Microsoft.

for more Information

https://docs.microsoft.com/en-us/power-bi/developer/api-automatic-retention-policy-for-real-time-dat...

If you Post the 200001 Row to your Dataset the first posted row will be dropped.

You can Modifiy the Retention policy with

POST https://api.powerbi.com/v1.0/myorg/datasets?defaultRetentionPolicy={None | basicFIFO}

If you set your streaming Dataset to none retention you can store up to 5 Million rows but you some have Limits. ( See below).

To POST Rows

• 75 max columns
• 75 max tables
• 10,000 max rows per single POST rows request
• 1,000,000 rows added per hour per dataset
• 5 max pending POST rows requests per dataset
• 120 POST rows requests per minute per dataset
• If table has 250,000 or more rows, 120 POST rows requests per hour per dataset
• 200,000 max rows stored per table in FIFO dataset
• 5,000,000 max rows stored per table in ‘none retention policy’ dataset
• 4,000 characters per value for string column in POST rows operation

https://docs.microsoft.com/en-us/power-bi/developer/api-rest-api-limitations

[Anonymous]

All datasets are created with the basicFIFO retention policy by default unless specified otherwise.

[Eric_Zhang]

@taoyang

It seems there's no way. Though the documentation in your link says

HEADERS
Content-Type:application/json
Location:https://api.powerbi.com/v1.0/myorg/datasets/{dataset ID}
BODY
{
  "@odata.context": "https://api.powerbi.com/v1.0/myorg/$metadata#datasets/$entity",
  "id": "7c0b090e--172874c749e0",
  "name": "SalesMarketing",
  "defaultRetentionPolicy": "basicFIFO"
}

I actually get below result when I call that request.

You can submit your request at Power BI Ideas and vote it up.

[jarvis_chen]

I have the same issue on setting basicFIFO on creating new data set following the example here:

http://docs.powerbi.apiary.io/#reference/datasets/datasets-collection/create-a-dataset

I do not see any output from the HTTP request showing the dataset is indeed created as basicFIFO. I have tried to push 200,000 into one dataset created with defaultRetentionPolicy = basicFIFO. However, it does not seem to work, and the dataset keeps growing even after hitting 200,000 marks. So far, I have not found any way to resolve this.

[Anonymous]

@jarvis_chen 

I don't think its a hard limit on 200 k. I've seen the dataset grow as much as 211k before purging records and drop back to 206 k.

[taoyang]

@Eric_Zhangthanks for your help 🙂