Forum Discussion

Sidhant's avatar
Sidhant
Icon for Advocate V rankAdvocate V
1 year ago

Managing and Exporting High-Volume Datasets (~1M Records) in Power BI

Hello everyone,
I'm currently working on a use case where I need to manage and export high-volume datasets (~1 million records) using Power BI, and I'm exploring multiple approaches. I’d appreciate your feedback on the current methods I’ve tried and would love to hear if there are better alternatives or optimizations.


Problem Statement:
Effectively manage and export large datasets (~1M records) in Power BI, while allowing users to select/deselect fields dynamically and ensuring smooth integration with Power Automate and Fabric for downstream processes.

 

Current Approaches:

Approach 1: Field Parameters + Power Automate + Paginated Reports

  • Using field parameters in Power BI to allow dynamic column selection.

  • Passing selected fields to Power Automate, which triggers a Paginated Report (RDL).

  • Logic in RDL is set up to show/hide columns based on parameters.

Issue:

The show/hide logic is being overridden — despite user selection, all columns are getting displayed. It seems the parameters are not being passed or consumed correctly within the RDL file.

 

Approach 2: Microsoft Fabric Lakehouse + Semantic Model:

 

  • Created a Lakehouse in Fabric to handle large data volumes efficiently.

  • Built a semantic model on top of the Lakehouse.

  • Developed reports using this model in Power BI.

  • Trying to trigger export or automation via Power Automate using Fabric data.

Issue:

Getting a "Bad Request" error while trying to integrate Power Automate with Fabric. Details of the error aren't very descriptive, so it's hard to debug.

 

I had couple of questions which are as follows:

 

  • Are these approaches going in the right direction for large dataset export scenarios?

  • Has anyone successfully implemented field-level selection with RDL exports based on Power BI parameters? How did you overcome the column visibility issues?

  • Any known limitations or best practices for using Power Automate with Fabric Lakehouse or Semantic Models?

  • Are there any alternative approaches or workarounds you’d recommend for:

    • Efficiently exporting 1M+ rows

    • Allowing dynamic field selection

    • Maintaining performance and scalability.

Any insights, samples or even partial suggestions would be highly appreciated. I’m open to reworking my approach if there’s a more scalable or reliable pattern others have used successfully.

Thanks in advance,
Sidhant

 

 

44 Replies