AdlsGen2ForbiddenError on Fabric Data Warehouse staged copy for one large table (after ~1 hour)

[sivanarayanan]

The problem

• The copy activity runs for about 1–1:15 hours.

• After that, it fails with AdlsGen2ForbiddenError on a CreateFile operation to the staging path.

• Other tables using the same pipeline, same gateway, same warehouse complete fine.

The Error:

ErrorCode=AdlsGen2ForbiddenError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ADLS Gen2 failed for forbidden: Storage operation 'CreateFile' on container '3ecef9a6-62b2-4e26-ab1d-7ab06ebdf88d' and path 'd44567c4-9f0c-4a59-bd31-fae5042b6813/Staging/d70c3422-7df6-4272-9200-1b321424caf2/MSSQLImportCommand/data_d70c3422-7df6-4272-9200-1b321424caf2_fc4bccf8-9d22-46cc-bd42-1ce681e4c14b.parquet' get failed with 'Operation returned an invalid status code 'Forbidden''. Possible root causes: (1). It is possible that the IP address of the self-hosted Data Factory runtime machines are not allowed by your Azure Storage firewall settings. (2). If the self-hosted Data Factory runtime use proxy server, it is possible that the IP address of the proxy server is not allowed by your Azure Storage firewall settings.. Account: ''. FileSystem: '3ecef9a6-62b2-4e26-ab1d-7ab06ebdf88d'. Path: 'd44567c4-9f0c-4a59-bd31-fae5042b6813/Staging/d70c3422-7df6-4272-9200-1b321424caf2/MSSQLImportCommand/data_d70c3422-7df6-4272-9200-1b321424caf2_fc4bccf8-9d22-46cc-bd42-1ce681e4c14b.parquet'. ErrorCode: 'AuthenticationFailed'. Message: 'Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.'. RequestId: 'f09508a5-101f-0030-4444-6658f5000000'. TimeStamp: 'Sat, 06 Dec 2025 00:07:28 GMT'..,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Operation returned an invalid status code 'Forbidden',Source=,''Type=Microsoft.Azure.Storage.Data.Models.ErrorSchemaException,Message=Operation returned an invalid status code 'Forbidden',Source=Microsoft.DataTransfer.ClientLibrary,'

Is there any known issue or limit around long-running staged copy to Fabric DW (e.g., token lifetime to the internal staging store)?

Any pointers, similar experiences, or official guidance would be really appreciated.

@v-menakakota  

[wardy912]

Hi @sivanarayanan 

The error is most likely caused by credential/token expiry. Common triggers include short-lived SAS tokens (default ~1 hour) or OAuth tokens that aren’t refreshed mid-run. Other tables succeed because their runs finish before expiry.

Try splitting the pipeline down using a ForEach so that you can use the token for multiple copy activities at the same time, or paramterise so that the token refreshes after a certain time.

--------------------------------

I hope this helps, please give kudos and mark as solved if it does!

Connect with me on LinkedIn.

Subscribe to my YouTube channel for Fabric/Power Platform related content!

[nielsvdc]

Hi @sivanarayanan, Fabric pipelines use Azure Storage SAS tokens or OAuth tokens for staging in ADLS Gen2. These tokens typically have a limited lifetime, commonly 1 hour by default for SAS tokens. If your copy activity runs longer than that, the token may expire before the staging write completes, causing the Forbidden error. This aligns with your problem: the failure occurs after ~1 hour.

You can solve this in a couple of ways:

1. Create a shortcut to the storage in a Lakehouse and use this as your source.
2. Use a service principal for authentication to the storage account instead of a SAS token or OAuth.
3. Use a Copy Job instead of a pipeline with a copy activity. A Copy Job uses parallelism and bypasses intermediate staging of the data before ingesting it into you warehouse.
4. Consider splitting the copy actvity into smaller batches to keep each copy activity process under 1 hour.

Hope this helps. If so, please give kudos 👍 and mark as Accepted Solution ✔️ to help others.