Forum Discussion

Lucy_Lim's avatar
Lucy_Lim
Regular Visitor
5 years ago

Recommendation to report on Salesforce historical data

Hello,

 

I was wondering if anyone could point me in the right direction here. I need to build a report on Power BI with historical data coming from Salesforce. 

 

The Objective: Create a Power BI report using data from Salesforce where data needs to be "appended" every month to the dataset

 

What I've tried:

  1. Connect to Salesforce with Power BI and retrieve my Salesforce data, however, as the Salesforce Reports API has a restriction that only supports up to 2,000 rows of data that's not a solution for me.
  2. Using CSV plus append in Power BI- I exported data from Salesforce to a CSV file (3YearData), that contains data for the past 3 years and built the report in Power BI using that CSV. Then every month I would connect to Salesforce and append the data (MONTH1) for that given month in Power BI report. The problem is that the following month (MONTH2), as the CSV file (3YEARDATA) does not hold MONTH1, I have a gap for that month.

 For option 2. is there a way that I can write the MONTH1 data into the CSV file?

 

Possible Alternative:

Another option that I could try is using data from SQL. The data I need has been run and saved into a SQL database up until now, however the code is outdated and it would required some work. But if using a SQL connection is easier, I could try doing that.

 

Any suggestion please:

Could someone please advise if you have experience with something similar to this? 

 

Thank you so much in advance!

3 Replies

  • Hi Lucy_Lim I know this reply is very late, but were you able to find a solution? I've used the Salesforce connector but I had some issues updating my data too. As a workaround, maybe you can try to test your connection with a 3rd party connector. I've tried windsor.ai, supermetrics and funnel.io. I stayed with windsor because it is much cheaper so just to let you know other options. In case you wonder, to make the connection first search for the Salesforce connector in the data sources list:

     

     

    After that, just grant access to your Salesforce account using your credentials, then on preview and destination page you will see a preview of your Salesforce fields:

     

     

    There just select the fields you need. It is also compatible with custom fields and custom objects, so you'll be able to export them through windsor.  Finally, just select PBI as your data destination and finally just copy and paste the url on PBI --> Get Data --> Web --> Paste the url.

     

     

  • Hi Lucy_Lim 

     

    For reporting on Salesforce historical data in Power BI, you might want to try an alternative connection method - Power BI Connector for Salesforce on AppExchange. It's a native Salesforce app installed inside your org, which makes pulling history objects (like Opportunity History, Field History, etc.) and large historical datasets more manageable:

     

    Why it might fit:

    Happy to set up a quick demo if it'd help. Just reach out via the support link.

     

    Cheers,

    Metrica Team.

  • Hi Lucy_Lim 

    The reason you are hitting that gap in your current approach is that Power BI is structurally designed to be a data-rendering tool rather than a state-retaining database; every single time your report undergoes a refresh, Power Query completely wipes its existing cache and evaluates the raw M-code instructions from a fresh slate, meaning your temporary "MONTH1" append query evaporates because it cannot write data backwards into your static 3YearData CSV file. Since you noticed that the Salesforce Reports API truncates your extractions at a hard 2,000-row limit, you should completely abandon the "Reports" folder within the connector window and instead connect directly to the raw Salesforce Objects layer (such as the Opportunity, Account, or Contact backend tables), which uses the high-scale Salesforce SOAP/REST API and completely bypasses that 2,000-row truncation limit so you can pull hundreds of thousands of historical rows natively. Once you point your query to the base Objects rather than a pre-built Salesforce user report, you can easily implement Power BI Incremental Refresh directly inside the Service by setting up a RangeStart and RangeEnd datetime parameter to automatically freeze your historical years in independent cloud partitions while only scanning Salesforce for the current month's changes during scheduled updates. If you choose not to use the native Object connector and prefer a completely bulletproof enterprise architecture, migrating this process to your existing SQL Server database is absolutely the best path forward; you can quickly fix that legacy script to run on a SQL Agent scheduled job that pulls the monthly delta from Salesforce and runs an INSERT INTO command to continuously append new rows onto a master tracking table, leaving Power BI with nothing to do except run a highly optimized, standard import query against a single, reliably growing database source.