Forum Discussion
Redshift Copy through Pipelines
Redshift is an option on pipelines, but it doesn't seem to allow you the option to choose a gateway? You can when ingesting via a dataflow but obviously I'd rather use copy data.
Am I missing something here? I can't ingest directly from redshift without a gateway, as we'd need to whitelist all the fabric IPs which would expose us to anyone else who would try to connect using Fabric
Hi JosephParkinson ,
You're right , Redshift in Fabric data pipelines doesn’t currently support using a gateway, which means it needs to connect over the internet. That would require you to whitelist Microsoft’s public IPs, which isn’t ideal from a security perspective.
Here are a few workarounds that you can try:
- Dataflows do support Redshift with a gateway. It’s not as flexible as pipelines, but it’s a secure way to pull data without opening up Redshift to the internet.
- If you're open to using Azure services, setting up a self-hosted integration runtime in ADF gives you better control over security and network access.
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Regards,
B Manikanteswara Reddy
6 Replies
Hi JosephParkinson ,
Thank you for reaching out to Microsoft Fabric Community.
It looks like the issue you're facing is very similar to one that another user encountered earlier. That issue has been resolved, and the solution might be helpful in your case as well.
Please refer to the post linked below for the full explanation and steps:
Solved: Pipeline - Copy Data - Amazon Redshift - No gatewa... - Microsoft Fabric Community
and also please refer the below document.Amazon Redshift connector overview - Microsoft Fabric | Microsoft Learn
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Regards,
B Manikanteswara Reddy
I don't think this does resolve my issue I'm afraid. I can't not use a gateway for the security reasons I outlined as it would require whitelisting publicly accessible IPs - or is there an alternative?
Hi JosephParkinson ,
You're right , Redshift in Fabric data pipelines doesn’t currently support using a gateway, which means it needs to connect over the internet. That would require you to whitelist Microsoft’s public IPs, which isn’t ideal from a security perspective.
Here are a few workarounds that you can try:
- Dataflows do support Redshift with a gateway. It’s not as flexible as pipelines, but it’s a secure way to pull data without opening up Redshift to the internet.
- If you're open to using Azure services, setting up a self-hosted integration runtime in ADF gives you better control over security and network access.
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Regards,
B Manikanteswara Reddy
Hi JosephParkinson ,
We wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?
If our response addressed, please mark it as Accept as solution and click Yes if you found it helpful.
Regards,
B Manikanteswara Reddy
