Prepaire a quit large dataset

[BF67]

Hi Guys,

my SQL Database has around 80 tables, with several dependencies.

there is a quite central table, lets call it "transaction". It has a "created" field in it. Nomen est Omen. This is intended to be the filter field for an incremental Update in the Fabric.

Now my Problem. PBI Desktop starts to update all tables, even if its itmes are not connected to an entry in the filtered "transaction" table. Result: the local Dataset has a around 11 GB size and can not be published.

How can I get a empty local dataset, or (whitout manual scripting depenencies) as set of data in a reasonable size?

Thank you for your hints.

Bernd   

[Hariharan_R]

Hi @BF67 

  Incremantal refresh parameters are best wayt to limit the records for development. If you are not able to do in that way then there are couple of ways.

1. Add Top or Limit the number of rows and parameterize that in Power Query and change the value in service.

2. if you are using view then change the view definition, add filter during the development in desktop and change that definition (remove filters) once it is published to service.

Thanks

Hari

[rajendraongole1 · Accepted solution]

Hi @BF67  - we can use parameter-driven filtering for development to limit the row count.

some reference links for incremental refresh:

Incremental refresh for semantic models in Power BI - Power BI | Microsoft Learn

How to Load Large Datasets in Power BI Desktop Using Parameters | phData

Hope this helps.