Forum Discussion

MAlimalik's avatar
MAlimalik
Regular Visitor
1 year ago
Solved

Big Size Power Query For Power BI Dashboard

Hi Guys, Need urgent help.   I am preparing a dashboard related to sales and revenue. For each month we have separate SCV file size around 1.5GB. I uploaded all 12 files of 2024 in the power query...
  • v-dineshya's avatar
    1 year ago

    Hi MAlimalik ,

    Thanks for reaching out to the Microsoft fabric community forum.

    Here are some best practices to optimize your dashboard:

    Optimize Data Load in Power Query
    Filter Early: Apply filters as early as possible in your query steps to reduce the amount of data being processed. Remove Unnecessary Columns: Remove columns you don’t need before appending or performing transformations. Disable Data Preview: In Power Query options, disable data preview to speed up the load process.

    Data Reduction Techniques
    Aggregate Data: If possible, aggregate data at the source level (e.g., summarizing daily sales to monthly totals before importing). Use Incremental Refresh: Power BI’s incremental refresh can help load only new or changed data instead of reprocessing the entire dataset.

    Data Model Optimization
    Star Schema Design: Use a star schema with fact and dimension tables. This reduces complexity and improves performance.
    Optimize Relationships: Keep the number of relationships to a minimum and avoid bi-directional filtering unless necessary.

    Hardware & System Performance
    Use 64-bit Power BI Desktop: Ensure you’re using the 64-bit version to take advantage of more RAM. Increase Memory Allocation: Adjust memory settings in Power BI to allocate more resources if possible.

    Query Folding
    Leverage Query Folding: Ensure transformations are pushed back to the data source when possible. This minimizes data processing in Power BI.

    Data Compression
    Optimize Data Types: Convert data types to more efficient formats (e.g., using integers instead of text where possible). Reduce Cardinality: Minimize unique values in columns, as high cardinality slows down performance.

    Batch Processing (Advanced)
    Split Files: If feasible, split large files into smaller chunks for easier processing. Load in Batches: Use parameters to load data in batches (e.g., process data for 3 months at a time).

    Can you please refer below article and threads.

    https://learn.microsoft.com/en-us/power-bi/guidance/power-bi-optimization
    https://community.fabric.microsoft.com/t5/Service/Best-way-to-handle-large-dataset-in-Power-BI/m-p/3584094
    https://community.fabric.microsoft.com/t5/Desktop/Power-BI-Development-and-Best-Practices/td-p/4632232

    If you find this post helpful, please mark it as an "Accept as Solution" and consider giving a KUDOS. Feel free to reach out if you need further assistance.
    Thanks and Regards