Data Refresh

Question

Hello all,&nbsp;I am new to Power BI and I'm trying to better understand how the data refresh function works. I am currently working on a report that shows the stock inventory history. I want my report to refresh every day so that the data and information is always up to date.&nbsp;&nbsp;My data source includes a CSV file that I manually export to my computer and manually import into Power BI. To get up to date information on the stock inventory, I have to manually export the CSV file everyday. This file does not update on its own.&nbsp;&nbsp;The only way I can think of doing this, is manually exporting the CSV file and then changing the name of the file every time.&nbsp;I've watched some tutorial videos, and they mention using a gateway connection and so on. But again, I'm new to Power BI and I'm just not sure how that is possible, when the file containing the stock inventory data isn't created until an export is made. (I hope this makes sense)&nbsp;Is a scheduled data resfresh even possible, since everything is being done manually?

ray_aramburo · Answer

There are 2 stages of refresh as I see it:
1) If you are working locally (this means only through Power BI Desktop) no matter how times your data source is refreshed you'll always need to manually refresh from Power BI Desktop to get the latest data.
2) If/When you publish your report to the Power BI Service, you can enable a scheduled refresh. Since you are working with csv files, I would recommend you to host them in a OneDrive/SharePoint/Cloud location so you can avoid using any gateway and then use the SharePoint folder or Web connection to access them.&nbsp;
&nbsp;
Now, you mention "the file containing the stock inventory data isn't created until an export is made." You'll need to identify what's exactly the origin of the data that builds the csv, is it a database?, ERP?, existing report?. Once identified, instead of building csv files you'll aim to connect to that data directly.&nbsp;

anonymous · Answer

Regarding your first point, I am working locally in the Power BI Desktop. I am not as concerned about getting the latest data in the desktop as I am about having the latest data on the Power BI Web Service.

I've already published my Product Stock Inventory report to the web service and have enabled a schedule refresh. When hosting the CSV file in OneDrive, would I still need to manually export the CSV file, and change the name to match the original, to my OneDrive to get the latest data?

Do you have any other clearer intructions on how to complete this?

ray_aramburo · Answer

So, if I understood correctly, your issue relies that the data source needs to be manually updated on daily basis, and because of this, it has a different name every day. Is that correct?
If that's the case, could you specify from where do you need to extract the csv files?

anonymous · Answer

Yes our product stock inventory is hosted on a website and the data from that can then be exported into a CSV file. Meaning that I have to manually export a CSV file everyday to get the latest data. And yes these CSV files all have a different name, with the date and timestamp that it was created in the name.&nbsp;&nbsp;&nbsp;

ray_aramburo · Answer

Okay, I see 2 options, one a short term workaround, and the other one as a long term solution:
1) Short-term: When you export the file, save it with a specific naming convention on a cloud-based location (Sharepoint/OneDrive/Google Drive). You will always name it this way on a daily basis. Since it will have the same name, the last file would be replaced/overwritten. Your Power BI report will have no issue because it will always read the file with the same name.
You can reduce some manual steps here if you develop some small automation in Power Automate to do the file saving process, but this solution will always inherently be manual because it requires for you to extract data manually.
2) The long-term solution is you finding a way to connect to the web page (or the database/data sources) behind it. The nature of this will depend on the architecture of the web page. By setting up a direct connection you avoid exporting manually the data. As to how to achieve this, my answer is, it depends on how is the data behind the web page structured/located.

Forum Discussion

Data Refresh

9 Replies

Related content

Data Refresh Errors

Refresh data

Data refresh error

Data refresh failing due to account refresh limit (Pro)

Data Refresh Failed