Forum Discussion
Power BI Data load , Refresh from Salesforce is Taking Too Long
Hi Community,
I'm currently working on a Power BI project where the data source is Salesforce. The dataset currently consists of only 3 Salesforce objects, with a total of approximately 5 million (50 lakh) rows. The data volume is expected to grow over time.
To optimize refresh performance, I've already implemented the following:
- Created partitions using Tabular Editor.
- Configured monthly incremental refresh.
- Processing the partitions using SQL Server Management Studio (SSMS).
However, the refresh process is still taking much longer than expected. Even processing only the latest monthly partition requires a significant amount of time.
I'm trying to understand what could be causing this performance issue.
Here are a few details about my setup:
- Data Source: Salesforce Objects
- Number of Tables: 3
- Current Row Count: ~5 million
- Storage Mode: Import
- Partitioning: Monthly partitions created using Tabular Editor
- Processing: SSMS (XMLA endpoint)
- Expected Future Growth: Data volume will continue to increase.
I would appreciate your guidance on the following questions:
- Is this refresh duration expected when using Salesforce as the source?
- Are there any known limitations or performance bottlenecks with Salesforce connectors?
- What factors should I investigate (network latency, Salesforce API limits, query folding, transformations, etc.)?
- Are there any best practices for improving refresh performance with Salesforce datasets of this size?
- Is monthly partitioning sufficient, or would a different partitioning strategy (weekly/daily) improve performance?
- Has anyone worked with Salesforce datasets of similar size, and what refresh times are considered reasonable?
Any recommendations, troubleshooting steps, or best practices would be greatly appreciated.
Thank you!
Hi sahgir123,
Thank you for reaching out to Microsoft Fabric Community.
Thank you Kagiyama_yutaka and parry2k for the prompt response.
Since you have already implemented incremental refresh and partitioning, this does not looks like a power bi model issue. The refresh time is more likely related to retrieving the data from salesforce through the salesforce API.
If the refresh is still taking a long time even for the latest partition, please check whether the delay is happening while retrieving the data from salesforce. If so, there is limited optimization available from the power bi side. Use a staging database or fabric lakehouse or warehouse as the refresh source for larger salesforce datasets.
Thanks and regards,
Anjan Kumar Chippa
9 Replies
- parry2k
Super User
sahgir123 here is a good starting point on how you can do it. Tweak and update the solution as per your need.
- parry2k
sahgir123 I would check the SF API limit. One thing: do you have a premium capacity? Based on your post, it looks like you do. If that is the case, why not bring the data incrementally into the fabric lakehouse and then use it in Power BI from there? This way, you don't have to deal with all this management, and you'll only pull incremental data.
Could you please provide detailed guidance on the implementation process? Specifically, I'd like to understand how we can use Microsoft Fabric in this scenario, as well as any alternative methods or best practices to improve performance.
- Kagiyama_yutaka
Continued Contributor
Power BI Import refresh from Salesforce slows down as the number of rows and columns increases. A practical way to improve refresh time is to reduce that volume by applying a date filter in Salesforce and keeping only the fields required by the model.
I have only 3 tables, each containing approximately 5 million (50 lakh) rows and 11 columns. I'm retaining only the last 7 months of data in the model.
v-achippaCommunity Support
Hi sahgir123,
Thank you for reaching out to Microsoft Fabric Community.
Thank you Kagiyama_yutaka and parry2k for the prompt response.
Since you have already implemented incremental refresh and partitioning, this does not looks like a power bi model issue. The refresh time is more likely related to retrieving the data from salesforce through the salesforce API.
If the refresh is still taking a long time even for the latest partition, please check whether the delay is happening while retrieving the data from salesforce. If so, there is limited optimization available from the power bi side. Use a staging database or fabric lakehouse or warehouse as the refresh source for larger salesforce datasets.
Thanks and regards,
Anjan Kumar Chippa
- metrica
Post Prodigy
Hi sahgir123,
Since even the latest partition is slow, first verify that the RangeStart and RangeEnd filters are actually limiting the data retrieved from Salesforce. Also keep only the required fields and avoid unnecessary concurrent Salesforce queries.
For datasets of this size, staging Salesforce data in a Fabric Lakehouse or Warehouse is also worth considering.
Power BI Connector for Salesforce provides filters, SOQL, field selection, and a documented incremental refresh setup to reduce the data exported.
AppExchange and 30-day trial:
https://appexchange.salesforce.com/appxListingDetail?listingId=31526f0e-abd8-4cb5-bd1a-3bd56b5c0577Incremental refresh guide:
https://metricasoftware.com/docs/salesforce/how-to-set-up-incremental-refresh-in-power-bi/Cheers,
Metrica Team
