Loading 18 million record in Power BI Pro license. How to handle performance.

[pbiforum123]

Loading 18 million of data in the Power BI Report and how to publish in Pro license when customer do not have budget for premium license? How will you handle the performance. Please note that they dont want to use Direct Query. 

[foodd]

Hi.  @Idrissshatila is conveying the truth.   Working within the limitations of

Professional when you really need Premium is challenging. The point of view is
to adjust the model or adjust the imported source upstream.

Question:  Is the data real-time or real-time+historical?

In that spirit, what are the barriers to downsizing the data model
at import or creating HOT and COLD Tables if the data represents
real-time+historical data?

Back to your original question that is one of performance rather than licensing.

Chris Webb in his November 2022 blogpost, 'How Much Data Can You Load Into Power BI?"

helps to explain the various limits on Power BGI dataset size and how to know if you are

likely to hit them.

Quoting Pat Mahoney who is a well know MS Employee and Forum member from 2020:

"It depends on your computer specs, but a typical laptop can handle a
model with 10s of millions of rows. The complexity and correctness
of your DAX expressions and the number of visuals/calculations on
the report page will dictate responsiveness to slicers, etc. And
the data source and the complexity of the data transformations
you do will govern your refresh time.

Power BI can handle billions of rows with the right setup. In demos,
they've shown >1 trillion rows."

[Idrissshatila]

Hello @pbiforum123 ,

You could use direct query when working with a huge dataset for a better performance.

check this out https://learn.microsoft.com/en-us/power-bi/connect-data/desktop-use-directquery

If I answered your question, please mark my post as solution, Appreciate your Kudos 👍

Follow me on Linkedin

[pbiforum123]

@Idrissshatila Sorry I forgot to mention that in the post. They dont want to use Direct query mode. How to handle using the import mode only in pro license?

[Idrissshatila]

Hello @pbiforum123 ,

Then you need to try to work this model as a star schema and see how it goes.

https://learn.microsoft.com/en-us/power-bi/guidance/star-schema

then try using incremental refresh https://youtu.be/Kui_1G6kQIQ

If I answered your question, please mark my post as solution, Appreciate your Kudos 👍

Follow me on Linkedin

[pbiforum123]

@Idrissshatila 

1. Model is already Star Schema only.

2. If my understanding is correct, incremental refresh is possible only in Premium license not Pro license.

[Idrissshatila]

Hello @pbiforum123 ,

Supported plans

Incremental refresh is supported for Power BI Premium, Premium per user, Power BI Pro, and Power BI Embedded datasets.

this is from microsoft documentation https://learn.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-overview

If I answered your question, please mark my post as solution, Appreciate your Kudos 👍

Follow me on Linkedin

[pbiforum123]

@Idrissshatila  You mean to say that using incremental model in pro license we can manage 18 millions records in Power BI report? Is that understanding corret?

[Idrissshatila · Accepted solution]

Hello @pbiforum123 ,

If you don't want Direct query, then this is your only solution, in incremental refresh the data won't re-import all 18 million records in every refresh, it will only re-import the records within a specific timeframe.

If I answered your question, please mark my post as solution, Appreciate your Kudos 👍

Follow me on Linkedin

[pbiforum123]

@Idrissshatila Even if it re-import the records within a specific timeframe. Can that handle 18 million records in Pro license? I mean will be able to publish the report which contains 18 millions records into workspace in pro license? Please let me know