Forum Discussion
Advocate IIIncremental Refresh fails at around 2 AM
Is the file date meaningful? Can it be used as part of the PK ?
Advocate IIHello lbendlin
I've did some tests and apparently it's not because of the datetime.localnow() or rows without a date.
I don't have a primary key in my table because this is data imported from files in Azure Data Lake. Each file contains multiple rows but they only differ based on a row which contains coordinates. What I do is creating an index column in Power Query M to make each row unique but apparantly this is the problem. When I'm archiving some data, the index is being regenerated which causes some rows having the same index (primary key).
As you can see in my image (I broke the relationships to not have the error during my incremental refresh but you see that I have the same primary keys for 2 different dates.
Is this possible? How do I create a primary key without having this issue?
Kind regards
Kind regards
Super UserIs the file date meaningful? Can it be used as part of the PK ?
- naelske_cronos
Hello lbendlin ,
That's a good idea. A concatenation of category and index was not enough like in the image. I've used some other fields like the datetime and the incremental refresh worked during the day change! As I only keep data for 7 days some rows are removed and new ones are added again, the 'Table.AddIndexColumn' always creates new index values sequentially and there was the problem of having two times 'Monitoring Exercise - 1691'.
Thanks again!
Kind regards