Forum Discussion

JosephParkinson's avatar
JosephParkinson
New Member
1 year ago
Solved

Redshift Copy through Pipelines

Redshift is an option on pipelines, but it doesn't seem to allow you the option to choose a gateway? You can when ingesting via a dataflow but obviously I'd rather use copy data.   Am I missing som...
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi JosephParkinson ,

     

    You're right , Redshift in Fabric data pipelines doesn’t currently support using a gateway, which means it needs to connect over the internet. That would require you to whitelist Microsoft’s public IPs, which isn’t ideal from a security perspective.

    Here are a few workarounds that you can try:

    1. Dataflows do support Redshift with a gateway. It’s not as flexible as pipelines, but it’s a secure way to pull data without opening up Redshift to the internet.
    2. If you're open to using Azure services, setting up a self-hosted integration runtime in ADF gives you better control over security and network access.

     

    If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!

    Regards,

    B Manikanteswara Reddy