I would like to build a database with data from our ATS.
The data in the ATS deletes after 4 months and I would like to build a data warehouse.
I have a live connection via html with our ATS and pull arround 11.300 data.
When I connect it via Power Query, only 5.000 data is loaded. Is it possible to get the full data?
Since the connection is live, when I lose data in out ATS, I also lose data in my Query.Is it possible to append data in a data-warehouse-structure?
When I figured out a way to append, is there a way I can merge rows with the latest data? So that if a certain column got an update, that the latest update is in the cell when t 2 rows merge?
Information:I use Office 365.
The headers are always the same.
I have 2 IDs I can merge to get a unique value with whom i can identify which ones are duplicates.
I hope there is a solution.
Thanks in advance for your help. I am looking forward to the discussion.
Hi @v-zhangti,thanks for your comment.I know how append and merge works. In Problem 3 I am trying to figure out a way to overwrite duplicates instead of deleting it.
Please refer to the following links in the hopes of helping you.Combine or Append Data in Power BI / Power Query: Main Concepts — The Power User
Solved: Correct way to append/merge data files - Microsoft Power BI Community
Community Support Team _Charlotte
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Take a look at the September 2023 Power BI update to learn more.
Join Microsoft Reactor and learn from developers.