Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Be one of the first to start using Fabric Databases. View on-demand sessions with database experts and the Microsoft product team to learn just how easy it is to get started. Watch now

Reply
Tim_Control
Regular Visitor

Data Collection / Incremental Refresh from Website - Is it Possible

Hi All,

 

I have been trying to establish a method for collecting data from a website.  Below is a link to the website.  Contained within this are a number of tables; "table 1" contains information on its course & speed, as well as when this information was last refreshed / updated.  For clarity when this is downloaded initially to Power Bi, it only creates a single row of data, which represents the vessels "live position"

https://www.vesselfinder.com/vessels/TANNER-IMO-0-MMSI-232038663

 

ideally, I would like to connect to this website, and re-download the latest course/speed information every 15 minutes, and save this information in a new row.

 

I have been able to Transform this data so far, as well as utilise "Use the first row as Headers" function.  However I'm currently unclear how to refresh it every 15 minutes (or another time interval) and continually add data to a new row.  I did try publishing the data and using a scheduled refresh, however this did not seem to work with "website" data, and also the scheduled refresh function appears to be better suited to scheduling in "days" rather than "minutes".

 

Any help which can be provided will be much appreciated.  I've just started using Power Bi today and am hoping to find a use for it.  But struggling at the moment.  So prescriptive guidance would be much appreciated.

 

Thanks,

Tim

1 REPLY 1
lbendlin
Super User
Super User

"ideally, I would like to connect to this website, and re-download the latest course/speed information every 15 minutes, and save this information in a new row."

 

Power BI has no memory. You need to do your data collection and archiving further upstream, for example via a curl script that appends the position to a text file on your PC. Then you can point Power BI to that text file.

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!

Dec Fabric Community Survey

We want your feedback!

Your insights matter. That’s why we created a quick survey to learn about your experience finding answers to technical questions.

ArunFabCon

Microsoft Fabric Community Conference 2025

Arun Ulag shares exciting details about the Microsoft Fabric Conference 2025, which will be held in Las Vegas, NV.

December 2024

A Year in Review - December 2024

Find out what content was popular in the Fabric community during 2024.