Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
3 years ago

Extend Database with live html connection

Hello Community,

 

I would like to build a database with data from our ATS.

The data in the ATS deletes after 4 months and I would like to build a data warehouse.

 

I have a live connection via html with our ATS and pull arround 11.300 data.

 

Problem 1:

When I connect it via Power Query, only 5.000 data is loaded. 
Is it possible to get the full data?

 

Problem 2:

Since the connection is live, when I lose data in out ATS, I also lose data in my Query.
Is it possible to append data in a data-warehouse-structure?

 

Problem 3:

When I figured out a way to append, is there a way I can merge rows with the latest data? So that if a certain column got an update, that the latest update is in the cell when t 2 rows merge?

 

Information:
I use Office 365.

The headers are always the same.

I have 2 IDs I can merge to get a unique value with whom i can identify which ones are duplicates.

 

I hope there is a solution.

Thanks in advance for your help. I am looking forward to the discussion.

 

Greetings

2 Replies