Forum Discussion
AdlsGen2ForbiddenError on Fabric Data Warehouse staged copy for one large table (after ~1 hour)
The problem
The copy activity runs for about 1–1:15 hours.
After that, it fails with AdlsGen2ForbiddenError on a CreateFile operation to the staging path.
Other tables using the same pipeline, same gateway, same warehouse complete fine.
The Error:
ErrorCode=AdlsGen2ForbiddenError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ADLS Gen2 failed for forbidden: Storage operation 'CreateFile' on container '3ecef9a6-62b2-4e26-ab1d-7ab06ebdf88d' and path 'd44567c4-9f0c-4a59-bd31-fae5042b6813/Staging/d70c3422-7df6-4272-9200-1b321424caf2/MSSQLImportCommand/data_d70c3422-7df6-4272-9200-1b321424caf2_fc4bccf8-9d22-46cc-bd42-1ce681e4c14b.parquet' get failed with 'Operation returned an invalid status code 'Forbidden''. Possible root causes: (1). It is possible that the IP address of the self-hosted Data Factory runtime machines are not allowed by your Azure Storage firewall settings. (2). If the self-hosted Data Factory runtime use proxy server, it is possible that the IP address of the proxy server is not allowed by your Azure Storage firewall settings.. Account: ''. FileSystem: '3ecef9a6-62b2-4e26-ab1d-7ab06ebdf88d'. Path: 'd44567c4-9f0c-4a59-bd31-fae5042b6813/Staging/d70c3422-7df6-4272-9200-1b321424caf2/MSSQLImportCommand/data_d70c3422-7df6-4272-9200-1b321424caf2_fc4bccf8-9d22-46cc-bd42-1ce681e4c14b.parquet'. ErrorCode: 'AuthenticationFailed'. Message: 'Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.'. RequestId: 'f09508a5-101f-0030-4444-6658f5000000'. TimeStamp: 'Sat, 06 Dec 2025 00:07:28 GMT'..,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Operation returned an invalid status code 'Forbidden',Source=,''Type=Microsoft.Azure.Storage.Data.Models.ErrorSchemaException,Message=Operation returned an invalid status code 'Forbidden',Source=Microsoft.DataTransfer.ClientLibrary,'
Is there any known issue or limit around long-running staged copy to Fabric DW (e.g., token lifetime to the internal staging store)?
Any pointers, similar experiences, or official guidance would be really appreciated.
v-menakakota
Hi wardy912 , nielsvdc
Thanks for your reply.
With my use cases,
wardy912, I cannot have multiple copy data activities at the same time within the ForEach
nielsvdc As I use built-in enable staging, I am unable to use a service principal for authentication, and splitting the copy activity into smaller batches will not work for me.
I solved the issue by using external storage instead of workspace storage.
5 Replies
- nielsvdc
Super User
Hi sivanarayanan, Fabric pipelines use Azure Storage SAS tokens or OAuth tokens for staging in ADLS Gen2. These tokens typically have a limited lifetime, commonly 1 hour by default for SAS tokens. If your copy activity runs longer than that, the token may expire before the staging write completes, causing the Forbidden error. This aligns with your problem: the failure occurs after ~1 hour.
You can solve this in a couple of ways:
- Create a shortcut to the storage in a Lakehouse and use this as your source.
- Use a service principal for authentication to the storage account instead of a SAS token or OAuth.
- Use a Copy Job instead of a pipeline with a copy activity. A Copy Job uses parallelism and bypasses intermediate staging of the data before ingesting it into you warehouse.
- Consider splitting the copy actvity into smaller batches to keep each copy activity process under 1 hour.
Hope this helps. If so, please give kudos 👍 and mark as Accepted Solution ✔️ to help others.
- wardy912
The error is most likely caused by credential/token expiry. Common triggers include short-lived SAS tokens (default ~1 hour) or OAuth tokens that aren’t refreshed mid-run. Other tables succeed because their runs finish before expiry.
Try splitting the pipeline down using a ForEach so that you can use the token for multiple copy activities at the same time, or paramterise so that the token refreshes after a certain time.
--------------------------------
I hope this helps, please give kudos and mark as solved if it does!
Connect with me on LinkedIn.
Subscribe to my YouTube channel for Fabric/Power Platform related content!
V-yubandi-msftCommunity Support
Hi sivanarayanan ,
If you get a chance, please review the response shared by wardy912 nielsvdc , it should address your concern.
Let us know if you need any additional information or clarification.Thanks for your valuable contribution, wardy912 nielsvdc .
Regards,
Yugandhar.Hi wardy912 , nielsvdc
Thanks for your reply.
With my use cases,
wardy912, I cannot have multiple copy data activities at the same time within the ForEach
nielsvdc As I use built-in enable staging, I am unable to use a service principal for authentication, and splitting the copy activity into smaller batches will not work for me.
I solved the issue by using external storage instead of workspace storage.- V-yubandi-msft
Thank you for the update. Glad to know the issue is clear now. Thanks for sharing.
