Mirrored SharePoint List (Preview) - “Failed to get document libraries” Error 404

Hi Tamanchu 

I managed to finalize yesterday an approach with the pipeline:

• LookupOldWatermark: reads the last processed date from the DW.
• Copy Activity: filters the SharePoint List using OData between the previous watermark and the TriggerTime, writing Parquet to the Lake House.
• Notebook: performs a MERGE of the Parquet into the Delta table using Id as the key, UPDATE if it exists, INSERT if it’s new. Returns the MAX(Modificado) of the processed data.
• SP UpdateWatermark: saves the MAX(Modificado) as the new watermark for the next run.

 A separate Copy Job was used to perform the initial full load, outside of the pipeline.

Thanks for your attention 🙂