Data flow issues

[joe4216]

Issue Summary We are experiencing consistent refresh failures in a Dataflow Gen2 that connects to Amazon Redshift through the On‑premises Data Gateway. The refresh attempts time out when the gateway tries to establish the connection.

Error Details

• Error Code: GatewayDataSourceOpenConnectionTimeout

• Request ID: b1d28483-5752-43ee-aac7-eb1bfdcf2e2f

• Message: “Something went wrong, please try again later. If the error persists, please contact support.”

Observed Behavior

• The Dataflow refresh starts normally in Fabric.

• The request is successfully passed to the gateway.

• The failure occurs when the gateway attempts to open a connection to Redshift.

• The error indicates a connection timeout rather than an authentication or query issue.

Expected Behavior The Dataflow Gen2 refresh should connect to Amazon Redshift through the configured gateway and complete without timing out.

Environment Details

• Data source: Amazon Redshift

• Connectivity: On‑premises Data Gateway

• Dataflow type: Dataflow Gen2

• Authentication method: [Specify]

• Gateway host location: [Specify]

• Redshift accessibility: [Specify – public endpoint or private VPC]

Troubleshooting Performed

• Verified that the Dataflow is correctly configured to use the gateway.

• Confirmed that the failure occurs during connection establishment, not during query execution.

• Attempted multiple refreshes with consistent failure.

• No recent changes were made to credentials or connection configuration.

Request We are seeking assistance to determine:

• Whether there are known issues or limitations with Dataflow Gen2 connecting to Amazon Redshift via the gateway.

• Whether any timeout, retry, or connection settings can be adjusted in Fabric or the gateway.

• What diagnostic logs or tools can help isolate the connection timeout between the gateway and Redshift.

• Whether the request is being correctly routed and handled by the Fabric service.

[v-hashadapu]

Hi @joe4216 , Hope you are doing well. Kindly let us know if the issue has been resolved or if further assistance is needed. Your input could be helpful to others in the community.

[v-hashadapu]

Hi @joe4216 , Thank you for reaching out to the Microsoft Community Forum.

We find the answer shared by @Ritaf1983  is appropriate. Can you please confirm if the solution worked for you. It will help others with similar issues find the answer easily.
Thank you @Ritaf1983  for your valuable response.

[Ritaf1983 · Accepted solution]

Hi @joe4216 

The following is a technical analysis of the GatewayDataSourceOpenConnectionTimeout error between Dataflow Gen2 and Amazon Redshift:

The timeout error indicates a failure during the initial TCP handshake or SSL negotiation phase, where the On-premises Data Gateway fails to establish a response from Redshift within the allocated timeframe. The first critical step is isolating the network layer from the Gateway server itself. Run the following command in PowerShell: Test-NetConnection -ComputerName [Redshift_Endpoint] -Port 5439. If this returns TcpTestSucceeded: False, the issue is a firewall blockage at the local server, corporate network, or within the AWS VPC Security Groups. Ensure that Redshift Inbound Rules explicitly allow traffic from the Gateway's public or private IP.

If basic connectivity is confirmed, analyze the Redshift connection logs by querying the stl_connection_log table. If no connection attempts from the Gateway IP are logged, the request is being dropped before reaching AWS. If attempts are logged with status failures, the issue may stem from cluster load or a mismatch in encryption protocols. Simultaneously, export the Gateway logs and search the Information and Error files for the specific Request ID. These logs will clarify if the failure is due to DNS resolution issues or a timeout in the Mashup Engine's connection pool.

In the context of Fabric and Dataflow Gen2, verify if the Gateway is overwhelmed by concurrent requests. If the server is under high resource pressure, you can increase the available containers by modifying the Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.dll.config file, specifically increasing the MashupDefaultPoolContainerMaxCount value. Furthermore, ensure the Gateway is updated to the latest version, as Fabric-specific optimizations for Dataflow Gen2 connection management are frequently released to address cloud-to-on-premises routing latencies.

Recommended Action Sequence:

Validate Port 5439: Use PowerShell on the Gateway host to confirm the path to Redshift is open.

Verify Redshift Logs: Check stl_connection_log to see if the handshake reaches the database.

Audit Gateway Resources: Check for CPU/Memory spikes and ensure the Gateway version is current.

Check SSL/TLS: Ensure the Gateway server supports the encryption level required by your Redshift cluster configuration.

If this post helps, then please consider Accepting it as the solution to help the other members find it more quickly

[rajendraongole1]

Hi @joe4216  -The issue is caused by network-level connectivity failure between the On-premises Data Gateway and Amazon Redshift.Resolution involves ensuring outbound connectivity from the gateway to the Redshift endpoint on port 5439, validating DNS resolution, and configuring firewall/security group rules or VPC connectivity (VPN/peering) where applicable.Once network access is established, the Dataflow Gen2 refresh should complete successfully without timeout errors.

Hope this helps.