Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The ultimate Microsoft Fabric, Power BI, Azure AI & SQL learning event! Join us in Las Vegas from March 26-28, 2024. Use code MSCUST for a $100 discount. Register Now

Reply
kenyonca
Frequent Visitor

Auto Refresh with multiple data sources

I am attempting to have my Power BI dashboard pull in data from multiple data sources and update the online dashboard autmoatically each night.

 

My first data source, SmartSheet, is updated daily and the revised data is scheduled to be pulled in each night.

 

The second data source is the real issue. Right now it is set up on an Excel sheet (approx. 38,000 rows).

I have tried to save it through a shared OneDrive. This however needs to have an online connection to ensure the gateway connection is successful. Unfortunately this isnt an option as I do not plan on having this computer always online for the 3 year duration of the project.

This second data source will rarely if ever change as it is just a long alignment approx 38km with associated longtitude and latitudes and a few columns of meta-data related to the stationing.

I attempted to use the "Enter Data" option but this kept on saying that there was too much data to be copied to the clipboard.

As well I tried to dump this into SmartSheet but it has a limit of 5000 rows.

 

Does anyone know another way to store these ~38,000 rows of data easily online so that PowerBI can automatically pull them in too without having to create a Gateway that would require my computer to be online for these connections.

 

Thanks!

3 REPLIES 3
Greg_Deckler
Super User
Super User

Why not just spin up an Azure SQL DB and load it there? Would be pretty cost effective. You could consider a SharePoint Online list but at 38,000 rows that might prove to be a problem. Azure Table storage might be even more economical.


@ me in replies or I'll lose your thread!!!
Instead of a Kudo, please vote for this idea
Become an expert!: Enterprise DNA
External Tools: MSHGQM
YouTube Channel!: Microsoft Hates Greg
Latest book!:
Mastering Power BI 2nd Edition

DAX is easy, CALCULATE makes DAX hard...

Yes that is an option we have thought about.

 

Is there no way to just either save the 38,000 lines in PowerBI with the Enter Data.

That or to get Power BI web version to not pull from that particular data source on the auto-refresh? But only pull from the SmartSheet data? The desktop version lets you choose to not include certain data sets when you trigger the refresh but the web version doesnt seem to have this capability.

@kenyonca,

 

Is there no way to just either save the 38,000 lines in PowerBI with the Enter Data.
Currently, we cannot do it by using Enter Data.

 

You could store your excel fiel to your Power BI group's OneDrive for Business. You can continue updating files you store in OneDrive for Business, and those updates are automatically reflected in the Power BI reports and dashboards based on the files. Please refer to the link below.
https://powerbi.microsoft.com/en-us/documentation/powerbi-service-connect-to-files-on-your-groups-on...

Regards,

Charlie Liao

Helpful resources

Announcements
Fabric Community Conference

Microsoft Fabric Community Conference

Join us at our first-ever Microsoft Fabric Community Conference, March 26-28, 2024 in Las Vegas with 100+ sessions by community experts and Microsoft engineering.

February 2024 Update Carousel

Power BI Monthly Update - February 2024

Check out the February 2024 Power BI update to learn about new features.

Fabric Career Hub

Microsoft Fabric Career Hub

Explore career paths and learn resources in Fabric.

Fabric Partner Community

Microsoft Fabric Partner Community

Engage with the Fabric engineering team, hear of product updates, business opportunities, and resources in the Fabric Partner Community.