Hello Community,
I would like to build a database with data from our ATS.
The data in the ATS deletes after 4 months and I would like to build a data warehouse.
I have a live connection via html with our ATS and pull arround 11.300 data.
Problem 1:
When I connect it via Power Query, only 5.000 data is loaded.
Is it possible to get the full data?
Problem 2:
Since the connection is live, when I lose data in out ATS, I also lose data in my Query.
Is it possible to append data in a data-warehouse-structure?
Problem 3:
When I figured out a way to append, is there a way I can merge rows with the latest data? So that if a certain column got an update, that the latest update is in the cell when t 2 rows merge?
Information:
I use Office 365.
The headers are always the same.
I have 2 IDs I can merge to get a unique value with whom i can identify which ones are duplicates.
I hope there is a solution.
Thanks in advance for your help. I am looking forward to the discussion.
Greetings
Hi @v-zhangti,
thanks for your comment.
I know how append and merge works. In Problem 3 I am trying to figure out a way to overwrite duplicates instead of deleting it.
Hi, @ManuelSAG
Please refer to the following links in the hopes of helping you.
Combine or Append Data in Power BI / Power Query: Main Concepts — The Power User
Solved: Correct way to append/merge data files - Microsoft Power BI Community
Best Regards,
Community Support Team _Charlotte
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.