Import data storage

[Anonymous]

Hello, 

I am trying to find a way to imported data to an appended data so that when the import refreshes, the appended table keeps growing. 

For example, 

there are two data tables, totalData and salesforceDate

the initial totalData data table looks like this:

Store	Item	Price
A	Chair	100
A	TV	475
A	Lamp	60

SalesforceData (Day 1)

Store	Item	Price
B	Phone	300
B	Toy	25
B	Candy	1

saleforceData (Day 2)

Store	Item	Price
C	Shoes	40
C	Bag	15
C	Shirt	15

If I set salesforceData to refresh everyday, 

My goal is so that at the end of Day 2, totalData (which appends salesforceData)  has data for stores A,B, and C (days 1 and 2). However, I am currently only getting data for stores A and C after day 2 as a result of the refresh deleting old data. Is there a way to work around this?

I need this because salesforce is only able to load reports of 2000 rows. Thus I need to pull data daily but do not have the time to append a new table every day. 

Thank you. Appreciate any solution to this problem. 

[metrica]

Hi @Anonymous 

Power BI Import refresh is not designed to work as append-only storage. On a full refresh, the imported table is reloaded from the source, so old Salesforce rows will not be kept automatically.

If you need daily history, I would use one of these patterns:

1. Store daily Salesforce extracts in a database, lakehouse, warehouse, or another append-capable storage layer.

2. Use incremental refresh only if your Salesforce query has a stable DateTime field and can be filtered by date.

3. If possible, use Salesforce Objects instead of Salesforce Reports and rebuild the logic in Power BI.

The 2,000-row issue comes from the Salesforce Reports path. If you keep using Salesforce Reports, daily appending can become a fragile workaround.

Another option is Power BI Connector for Salesforce by Metrica Software: https://appexchange.salesforce.com/appxListingDetail?listingId=31526f0e-abd8-4cb5-bd1a-3bd56b5c0577

It lets you prepare Salesforce data sources, select objects and fields, apply filters, preview the output, and use SOQL mode when needed. This can help avoid splitting Salesforce reports into manual daily exports.

Docs and support:
https://metricasoftware.com/docs/salesforce/
https://metricasoftware.com/docs/salesforce/contact-support/

Cheers,
Metrica Team

[QueryWhiz]

Hi @Anonymous were you able to find a solution? As a workaround, maybe you can try to test your connection with a 3rd party connector. I've tried windsor.ai, supermetrics and funnel.io. I stayed with windsor because it is much cheaper so just to let you know other options. In case you wonder, to make the connection first search for the Salesforce connector in the data sources list:

After that, just grant access to your Salesforce account using your credentials, then on preview and destination page you will see a preview of your Salesforce fields:

There just select the fields you need. It is also compatible with custom fields and custom objects, so you'll be able to export them through windsor.  Finally, just select PBI as your data destination and finally just copy and paste the url on PBI --> Get Data --> Web --> Paste the url. 

[v-xicai]

Hi @Anonymous ,

There is a workaround you can considerate, following the steps below.

1.Save these data tables in OneDrive for Business, create a new column named "Day number" in totalData table to storage the days in later step 3.

2.Connect to the excel file totalData located in OneDrive for Business using Web connector, see more detail: Use OneDrive for Business links in Power BI Desktop.

3.Once there are new SalesforceData, you can append the new data into the totalData table by doing copy and post.

4.Click Refresh button to get the latest data.

Best Regards,

Amy

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly

[Anonymous]

Thank you for your help. @v-xicai 

So does the salesforce data have to be copy and posted every day manually? Also, just for clarification, is  the salesforce data being appended on one drive?

[v-xicai]

Hi @Anonymous ,

It is a clumsy workaround, right? You may need copy and poste every day manually. For the SalesforceData, you may needn't to upload it to OneDrive for Business.

Best Regards,

Amy