Forum Discussion
Write into Google Cloud Storage from Copy Job/Copy Activity within Fabric
Hi all,
I’m setting up a Microsoft Fabric Copy Activity to upload CSVs from our Lakehouse into a third-party Google Cloud Storage bucket. The bucket is owned by them, they upload data daily for us to ingest, and now we need to send files back for their processing.
They have provided us with HMAC key and a secret. The role they've granted us is Storage Object Admin.
I've tried setting it up using both Copy Job and Copy Data activity within a pipeline but both fail with the same errors.
What works
Using the provided HMAC Access ID/Secret, we can list and read files via a shortcut.
The connection only validates when using the bucket-specific URL:
https://<bucket-name>.storage.googleapis.com
What doesn’t
Using the recommended endpoint https://storage.googleapis.com fails with:
Expression.Error: Invalid UrlUsing the bucket URL connects, but browsing or writing into it fails with:
TrustFailure: Could not establish trust relationship (invalid certificate)
or Access denied (403)
Tried so far
Confirmed HMAC auth works for reads.
Tested both path-style and virtual-hosted URLs.
Need help with
Any other permissions required beyond Viewer/Creator?
Examples of successful GCS writes via HMAC from Fabric would be great.
Thanks!
my view is the issue is not with Fabric’s copy activity itself but with GCS auth and endpoint configuration. Also, fabric currently has limited support for GCS writes using HMAC authentication. The issue you are seeing, “403 Forbidden” or “TrustFailure” happens because Fabric’s copy activity expects an S3 compatible endpoint with a valid SSL certificate, while GCS’s signed certificate only matches storage.googleapis.com, not bucket-specific URLs.
Key points to know:
Supported setup
Use S3 connector in Fabric with the GCS HMAC key/secret.
Set endpoint to https://storage.googleapis.com.
Enable virtual-hosted-style addressing (<bucket>.storage.googleapis.com) only if SSL trust is established (not always valid for 3rd party buckets).
Required permissions in GCS
roles/storage.objectAdmin (read/write/delete on objects).
Ensure the HMAC key belongs to a service account tied to that role, not a user account.
Workaround for writes
If direct write fails, stage files in Fabric Lakehouse or ADLS, then push to GCS using a Notebook (Python + boto3) or custom REST API with the HMAC signature.
In short, Fabric copy activity can read from GCS via HMAC, but write support is unreliable unless done through a notebook using an S3-compatible client like boto3.
3 Replies
- Vinodh247
Super User
my view is the issue is not with Fabric’s copy activity itself but with GCS auth and endpoint configuration. Also, fabric currently has limited support for GCS writes using HMAC authentication. The issue you are seeing, “403 Forbidden” or “TrustFailure” happens because Fabric’s copy activity expects an S3 compatible endpoint with a valid SSL certificate, while GCS’s signed certificate only matches storage.googleapis.com, not bucket-specific URLs.
Key points to know:
Supported setup
Use S3 connector in Fabric with the GCS HMAC key/secret.
Set endpoint to https://storage.googleapis.com.
Enable virtual-hosted-style addressing (<bucket>.storage.googleapis.com) only if SSL trust is established (not always valid for 3rd party buckets).
Required permissions in GCS
roles/storage.objectAdmin (read/write/delete on objects).
Ensure the HMAC key belongs to a service account tied to that role, not a user account.
Workaround for writes
If direct write fails, stage files in Fabric Lakehouse or ADLS, then push to GCS using a Notebook (Python + boto3) or custom REST API with the HMAC signature.
In short, Fabric copy activity can read from GCS via HMAC, but write support is unreliable unless done through a notebook using an S3-compatible client like boto3.
Thank you, Vinodh. Appreciate the detailed breakdown and the workaround suggestion.
Hi mara5572
Were you able to set this up and running? I need to implement same and need some help as well.
