Redshift Copy through Pipelines

[JosephParkinson]

Redshift is an option on pipelines, but it doesn't seem to allow you the option to choose a gateway? You can when ingesting via a dataflow but obviously I'd rather use copy data.

Am I missing something here? I can't ingest directly from redshift without a gateway, as we'd need to whitelist all the fabric IPs which would expose us to anyone else who would try to connect using Fabric 

[JosephParkinson]

I don't think this does resolve my issue I'm afraid. I can't not use a gateway for the security reasons I outlined as it would require whitelisting publicly accessible IPs - or is there an alternative? 

[v-bmanikante · Accepted solution]

Hi @JosephParkinson ,

You're right , Redshift in Fabric data pipelines doesn’t currently support using a gateway, which means it needs to connect over the internet. That would require you to whitelist Microsoft’s public IPs, which isn’t ideal from a security perspective.

Here are a few workarounds that you can try:

1. Dataflows do support Redshift with a gateway. It’s not as flexible as pipelines, but it’s a secure way to pull data without opening up Redshift to the internet.
2. If you're open to using Azure services, setting up a self-hosted integration runtime in ADF gives you better control over security and network access.

If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!

Regards,

B Manikanteswara Reddy

[v-bmanikante]

Hi @JosephParkinson ,

We wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?

If our response addressed, please mark it as Accept as solution and click Yes if you found it helpful.

Regards,

B Manikanteswara Reddy

[v-bmanikante]

Hi @JosephParkinson ,

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?

If our response addressed, please mark it as Accept as solution and click Yes if you found it helpful.

Regards,

B Manikanteswara Reddy

[v-bmanikante]

Hi @JosephParkinson ,

May I ask if you have gotten this issue resolved?

If it is solved, please mark the helpful reply or share your solution and accept it as solution, it will be helpful for other members of the community who have similar problems as yours to solve it faster.

Please don't forget to give a "Kudos " – I’d truly appreciate it!

Regards,

B Manikanteswara Reddy

[v-bmanikante]

Hi @JosephParkinson ,

Thank you for reaching out to Microsoft Fabric Community.

It looks like the issue you're facing is very similar to one that another user encountered earlier. That issue has been resolved, and the solution might be helpful in your case as well.

Please refer to the post linked below for the full explanation and steps:

Solved: Pipeline - Copy Data - Amazon Redshift - No gatewa... - Microsoft Fabric Community

and also please refer the below document.

Amazon Redshift connector overview - Microsoft Fabric | Microsoft Learn

If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!

Regards,

B Manikanteswara Reddy