How to efficiently load large dataset (>1GB) from source into Power BI

[SBC]

Hi Experts,

I have a situation with my Power BI report:

• I can only select entire tables from the source (no option to write SQL queries or apply filters at the source level).After fetching the data into Power BI, I can make transformations.
• I have a date column in two tables. If I filter only 3 days of data, it works fine. But if I try to pull all data, it’s huge (more than 1GB) and Power BI fails to load it.
• I’ve already removed unused columns and duplicate data in Power Query.

My requirement:
I need to bring this large dataset into Power BI efficiently so that I can work with historical data without exceeding file size limits.

Question:

• What is the best approach to handle this situation?
• Can incremental refresh or dataflows help here when I cannot filter data at the source?
• Are there other optimization techniques I should apply?

Thanks in advance for your help!

Regards,

SBC

[Shahid12523 · Accepted solution]

Don’t import full 1GB into Desktop — use Incremental Refresh with RangeStart/RangeEnd → Desktop loads only a slice, Service handles partitions.

Use Dataflows as a staging layer for heavy tables.

Keep recent data in detail, aggregate old data.

Optimize model (drop unused columns, use proper types).

If still too big → use Composite Model (Import recent + DirectQuery old).

Best option: Incremental Refresh + model optimization.

[v-achippa]

Hi @SBC,

Thank you @Shahid12523 for the response.

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided by Shahid for the issue worked? Is your issue resolved? or let us know if you need any further assistance.

Thanks and regards,

Anjan Kumar Chippa

[v-achippa]

Hi @SBC,

Thank you for reaching out to Microsoft Fabric Community.

Thank you @audreygerred for the prompt response.

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided by the user for the issue worked? or let us know if you need any further assistance.

Thanks and regards,

Anjan Kumar Chippa

[SBC]

Hi @v-achippa ,

Thank you for the follow-up.

The previous response didn’t fully address my expectations. I’m reviewing alternative approaches and best practices within our Power BI Pro license constraints. I’m also monitoring the community for any strong suggestions that align with the requirement. 

Thanks,
SBC

[audreygerred]

Hello! Power BI typically handles large data great and the compression in import mode is amazing. What Fabric SKU do you have - that determines your size limitations. If you are just using pro, your model cannot exceed 1 GB. F64 is 25GB per model, F128 is 50GB per model, F256 is 100GB per model.