Hello,
I am looking for some guideance on creating some code to load into my Power BI Data Set. I am trying to write some code simular to how I used it in Qlik. Basically I have two data folders. The first is my base data (old/historical) and this data set never changes. The second is my new data set and this gets updated weekly with new records and changed records.
How I want it to work is to load the old CSV data file and load in the CSV new. The new file will load in new/changed records and replace that with the data being loaded in from the old. This is based on the Gift system ID field, so anytime a Gift system ID has new/changed records in the new data file, it should delete all these rows based on Gift system ID in the old and replace them with the new data. This process is needed to avoid duplications.
In my example below I have added two data sources for example. The first being the old and the second being the new. As you can see there is changed data for the new data file (*** indicating change). I need the old data to be deleted and the new data to populate my data set.
I have also attached the code I am using in Qlik. If there is an easier way to do this please let me know. I am open to any options.
https://drive.google.com/drive/folders/13TJBe80sM88x1FmMhZURbN9Ivmn47diR?usp=sharing
| Old | | | | | |
| Gift System ID | Appeal ID | Campain ID | Fund ID | Package ID | Amount |
| 1 | 1 | 1 | 1 | 1 | 200 |
| 1 | 2 | 1 | 1 | 1 | 250 |
| 1 | 3 | 1 | 1 | 1 | 100 |
| 1 | 4 | 1 | 1 | 1 | 100 |
| 1 | 5 | 1 | 1 | 1 | 300 |
| 2 | 1 | 1 | 1 | 1 | 50 |
| 2 | 2 | 1 | 1 | 1 | 20 |
| 2 | 1 | 1 | 1 | 1 | 100 |
| | | | | | |
| | | | | | |
| New | | | | | |
| Gift System ID | Appeal ID | Campain ID | Fund ID | Package ID | Amount |
| 1 | 1 | 1 | 1 | 1 | 200 |
| 1 | 2 | 1 | 1 | 1 | 250 |
| 1 | 3 | 1 | ***2 | 1 | 100 |
| 1 | 4 | 1 | 1 | 1 | 100 |
| 1 | 5 | 1 | 1 | 1 | ***10000 |
