Forum Discussion
Mirrored SharePoint List (Preview) - “Failed to get document libraries” Error 404
Hello everyone,
I’m exploring the Mirrored SharePoint Online List (preview). I ran a test with a small list and it has been replicating changes correctly. However, I keep seeing the error 'led to get document libraries. Request failed with status code 404, x-ms-root-activity-id: …' on the screen.
Could this be happening because the feature is still in preview?
Thank you
Hi Tamanchu
I managed to finalize yesterday an approach with the pipeline:
- LookupOldWatermark: reads the last processed date from the DW.
- Copy Activity: filters the SharePoint List using OData between the previous watermark and the TriggerTime, writing Parquet to the Lake House.
- Notebook: performs a MERGE of the Parquet into the Delta table using Id as the key, UPDATE if it exists, INSERT if it’s new. Returns the MAX(Modificado) of the processed data.
- SP UpdateWatermark: saves the MAX(Modificado) as the new watermark for the next run.
A separate Copy Job was used to perform the initial full load, outside of the pipeline.
Thanks for your attention 🙂
7 Replies
Hi thaisstefani18_ ,
The "Failed to get document libraries. Request failed with status code 404" error in the Mirrored SharePoint List feature is typically caused by one of the following:
- Permissions issue Make sure the account you're using has at least Site Member (Edit) permissions on the SharePoint site. The Mirroring feature needs access to both the site and its document libraries. Check that:
- You have access to the SharePoint site directly (try opening it in a browser)
- The site URL is correct and not pointing to a subsite or hub site that redirects
2. Preview limitations Since Mirrored SharePoint List is still in Preview, there are known limitations:
- It may not work with all SharePoint site types (e.g., classic sites vs. modern sites)
- Some tenant-level settings might block the API calls that Fabric uses behind the scenes
- The feature requires the SharePoint site to be in the same tenant as your Fabric Workspace
3. Site URL format. Ensure you're using the correct format. The URL should be the root site URL (e.g., https://yourtenant.sharepoint.com/sites/yoursite), not a direct link to a list or library.
4. Admin consent Your SharePoint admin may need to grant consent for Fabric to access SharePoint APIs. Check with your admin if there are any conditional access policies or API restrictions in place.
Since this is a Preview feature, I'd also recommend opening a support ticket with Microsoft to report the 404 error it could be a bug they need to investigate.
Let me know if any of these suggestions help! give a kudo 😉
Hello Tamanchu
Thank you for your time 🙂
Permissions and access are all correctly configured.
I will continue to monitor this feature until it reaches GA and will consider opening a ticket with Microsoft.
- Hi thaisstefani18_,Thank you for confirming that you’ve raised a support ticket. Please share any updates here as they become available, as this will help other community users who may be experiencing a similar issue.
Opening a support ticket is a good idea the more feedback Microsoft gets on this Preview feature, the faster it will improve.
In the meantime, if you need a working solution right now, here are two alternatives that work well in production:
- OneLake Shortcut to SharePoint Instead of Mirroring, you can create a shortcut from your Lakehouse directly to the SharePoint document library. This gives you read access to the files without the Mirroring preview limitations.
- Pipeline with Copy Activity Set up a Data Factory pipeline that pulls data from SharePoint via the SharePoint Online connector. It's more manual than Mirroring, but it's fully GA and reliable:
- Source: SharePoint Online List or Document Library
- Destination: Lakehouse Delta Table
- Schedule: trigger on demand or on a schedule
I've been working on a similar SharePoint-to-Fabric ingestion pattern and plan to share a detailed walkthrough soon.
Keep us posted on your support ticket it would be great to know what Microsoft recommends!
Hello Tamanchu
- OneLake shortcut to SharePoint
As I understand, this is currently only available for SharePoint Folder, correct? - Pipeline with copy activity
I’m trying to implement an incremental approach since the list currently has nearly 2 million records. However, I’m facing some challenges due to source limitations. - Mirroring
At the moment, mirroring is not replicating all records from the source and also does not keep the selection limited to List, which could help avoid the 404 error.
Other attempts:
- Notebook: no API access at the moment
- Dataflow Gen2: failed due to memory limits
- Copy Job: returned an error (I’ve opened a topic in the community)
https://community.fabric.microsoft.com/t5/Fabric-platform/Copy-Job-2100-InternalServerError-on-SharePoint-List-gt-2M-Items/m-p/5136103#M26517
In the image below, even when selecting only the “List” option, the mirroring does not preserve this setting, it always ends up with everything selected.Thank you 🙂
Hi thaisstefani18_,
Hi @thaisstefani18_,Thank you for the detailed feedback these are real-world challenges that many of us face with large SharePoint lists. Let me address each point with references from the official documentation.1. Mirroring selecting everything instead of just "List"This is a known behavior in the Preview. According to the Mirrored SharePoint List documentation, mirroring currently replicates both Document Library metadata (via OneLake shortcuts) and managed table data (converted to Parquet/Delta). The selection UI doesn't yet allow granular filtering this is expected to improve as the feature moves toward GA.My recommendation: Open a support ticket referencing this behavior. Microsoft tracks Preview feedback to prioritize GA improvements. In the meantime, consider the workarounds below for your 2M+ records scenario.2. Pipeline Copy Activity Incremental loading for 2M+ recordsThe SharePoint Online List connector in Copy Activity does support OData queries for filtering. You can use the $filter parameter with the Modified column to implement incremental loading. From the Copy Activity documentation:Example OData query for incremental load:$filter=Modified ge datetime'2026-04-01T00:00:00Z'&$top=5000Recommended pattern Pipeline with watermark:1. Use a Lookup Activity to get the MAX(Modified) from your Lakehouse destination table2. Pass that value as a parameter to the Copy Activity's OData query property:$filter=Modified ge datetime'@{activity('GetWatermark').output.firstRow.MaxModified}'3. Set the destination to Append mode in your LakehouseImportant limitation: SharePoint has a URL length limit of ~2,100 characters for OData queries. Keep your $filter and $select expressions concise. (Troubleshooting reference)Also, increase the httpRequestTimeout beyond the default 5 minutes — for 2M+ records, consider setting it to 00:30:00 (30 minutes).3. Dataflow Gen2 Memory limit failuresFor very large SharePoint lists, Dataflow Gen2 can hit memory limits because OData filters are applied within Fabric after the data is received from the source (as noted in the incremental refresh documentation).Workaround using incremental amassing pattern:1. Create a query in your Dataflow that reads the MAX(ID) or MAX(Modified) from your Lakehouse destination2. Use that value to filter the source, loading only new/changed records3. Set the destination update method to AppendThis way, each refresh only processes a small batch instead of the full 2M records, staying within memory limits.4. Recommended production approach for 2M+ SharePoint records- Pipeline + Copy Activity with OData$filer : best for Incremental loads using Modifed column (GA)
- Dataflow Gen2 + Incremental : best for Small incremental batches with transformations (GA)
- Mirroring is : Wait for GA with better list selection & large list support (Preview)
The Pipeline approach is the most reliable for your scale.
Sources:
- https://learn.microsoft.com/en-us/fabric/mirroring/sharepoint-list
- https://learn.microsoft.com/en-us/fabric/data-factory/connector-sharepoint-online-list-copy-activity
- https://learn.microsoft.com/en-us/fabric/data-factory/connector-sharepoint-online-list-overview
- https://learn.microsoft.com/en-us/fabric/data-factory/tutorial-setup-incremental-refresh-with-dataflows-gen2
- https://learn.microsoft.com/en-us/fabric/data-factory/connector-troubleshoot-sharepoint-online-list
- https://learn.microsoft.com/en-us/azure/data-factory/connector-sharepoint-online-list?tabs=data-factory
Hope this helps!
- OneLake shortcut to SharePoint
