Forum Discussion

dunn1373's avatar
dunn1373
Frequent Visitor
1 year ago
Solved

PowerBI data model Refresh hung up - 3 hour failure

We have a few data models we publish that contain the working elements that populate a number of reports.  All but one of them are under 1 GB.  The data model that is over 1GB is in a Premium workspace.  Our primary data model (called "Common Data Model") has had intermittent issues about half of our days over the last 2-3 weeks.  Some days it refreshes in 40-50 minutes and some days it will continue to refresh for 2+ hours until it eventually fails around the 3 hour mark (gives us a latency/time out error).  Please don't give me some line like "have you looked at optimizing your table structure"...we have done that.  If the model itself was a problem, then it wouldn't refresh perfectly fine some days and fail on others.  Additionally, on days it fails, I'm able to refresh it on my local in ~1 hour.  So there isn't a data element that is failing.  

 

I've had this happen before and the only response I've had that made sense was that "there are too many PowerBI Pro instances using the same resources in PowerBI Service on some days so it ends up failing".  I can only surmise that is also what is happening here.  If it's something else I can do, I would love to know.  But if it's that, Microsoft should enable more resources to that.  We pay for the service and I shouldn't have to flip a coin every day as to whether it will work or not.  

  • Hi dunn1373 

    To improve performance and ensure data consistency in your Power BI model, especially when working with bitemporal and snowflake schemas, consider the following strategies:

    1. Flatten your schema where possible using Power Query or SQL views to reduce complexity and avoid issues with inner joins.
    2. Use a bridge date table (e.g., BusinessDate) with TREATAS() for handling From_Date and To_Date across multiple tables. This helps apply time-based logic without expanding rows unnecessarily.
    3. Split data using composite models — import recent data for performance and use DirectQuery or summarized tables for older records.
    4. Manage relationships with DAX: Keep certain relationships inactive and use USERELATIONSHIP() when needed to control filter context safely.
    5. Avoid DAX in slicers: Instead, create a proper dimension table for filtering to ensure it works with visuals.

    These techniques help keep your model scalable and responsive, even with complex logic and large datasets.

    If the above information helps you, please give us a Kudos and marked the Accept as a solution.

    Best Regards,
    Community Support Team _ C Srikanth.

8 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    I would open a ticket with mircosoft to investigate if you are at this point.

    You could always create a new workspace and deploy to new workspace, setup a new Data Source connection/authentication and see if it finishes in time.

  • Hi dunn1373 ,

     

    Change your data gateway and see whether the same issue occurs or not. If your workspace is on premium capacity you can check capacity metrics too.

     

    I hope it helps.

     

    Thanks,

    Sai Teja 

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi dunn1373 

    Thanks for bringing this to the Community.
    Please do follow the below points that might resolve your issue.

    • Implement Incremental Refresh on your largest tables so you only process new or changed partitions each day instead of the full model. That cuts your refresh workload dramatically.

    • Break big tables into partitions (by date or other key) and use the XMLA endpoint or the Premium Gen2 REST APIs to refresh only those partitions that need updating.

    • Install and review the Premium Capacity Metrics app to pinpoint CPU, memory or throttling spikes during your slow days—this confirms if you’re hitting capacity limits.

    • If you see consistent contention, enable autoscale on your Premium Gen2 capacity or scale up to a higher SKU to guarantee more compute headroom.

    • Stagger or reschedule heavy model refreshes to off-peak hours, or spread them across multiple capacity nodes so they don’t all compete at once.

    • As a last resort, consider moving this very large model to its own dedicated Premium capacity (or a PPU) to isolate its resource demands.

    These patterns ensure predictable refresh times rather than “flip-a-coin” performance.

    If this solves your issue, please give us Kudos and mark this as Accepted Solution.
    Best regards,
    Community Support Team _ C Srikanth.

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi dunn1373 

    We haven't heard from you since last response and just wanted to check whether the solution provided has worked for you. If yes, please Accept as Solution to help others benefit in the community.
    Thank you.

    If the above information is helpful, please give us Kudos and mark the response as Accepted as solution.
    Best Regards,
    Community Support Team _ C Srikanth.

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi dunn1373 

    I wanted to follow up since I haven't heard from you in a while. Have you had a chance to try the suggested solutions?
    If your issue is resolved, please consider marking the post as solved. However, if you're still facing challenges, feel free to share the details, and we'll be happy to assist you further.
    Looking forward to your response!

    Best Regards,
    Community Support Team _ C Srikanth.

    • dunn1373's avatar
      dunn1373
      Frequent Visitor

      I have taken actions of my own, but most of the proposed suggestions and/or solutions were not helpful.  I have found ways to slim the model and refactored some code to help but I know this will come up again.  My feedback to the Microsoft support would be as follows:

       

      1 - if you do incremental refresh, downloading the full data model and making changes is more difficult

      2 - we have broken down bigger tables into smaller partitions already - that's not really a solution to the issues I have operating in the PowerBI Pro environment.  you're asking me to make it easier on your product by making it more difficult for me.  I can, and will, continue to improve my data models.  but it shouldn't be the second suggestion.  that's like an auto manufacturer telling a driver to try driving slower.  

      3 - As i stated, this is not a Premium Capacity

      4 - Moving our schedules to not align with business needs is not a solution either - again you're asking me to ease the burden on Microsoft by making it less convenient for me and my stakeholders

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi dunn1373 

    To improve performance and ensure data consistency in your Power BI model, especially when working with bitemporal and snowflake schemas, consider the following strategies:

    1. Flatten your schema where possible using Power Query or SQL views to reduce complexity and avoid issues with inner joins.
    2. Use a bridge date table (e.g., BusinessDate) with TREATAS() for handling From_Date and To_Date across multiple tables. This helps apply time-based logic without expanding rows unnecessarily.
    3. Split data using composite models — import recent data for performance and use DirectQuery or summarized tables for older records.
    4. Manage relationships with DAX: Keep certain relationships inactive and use USERELATIONSHIP() when needed to control filter context safely.
    5. Avoid DAX in slicers: Instead, create a proper dimension table for filtering to ensure it works with visuals.

    These techniques help keep your model scalable and responsive, even with complex logic and large datasets.

    If the above information helps you, please give us a Kudos and marked the Accept as a solution.

    Best Regards,
    Community Support Team _ C Srikanth.

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    HI dunn1373 

    It's been a while since I heard back from you and I wanted to follow up. Have you had a chance to try the solutions that have been offered?
    If the issue has been resolved, can you mark the post as resolved? If you're still experiencing challenges, please feel free to let us know and we'll be happy to continue to help!
    Looking forward to your reply!


    Best Regards,
    Community Support Team _ C Srikanth.