March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Early bird discount ends December 31.
Register NowBe one of the first to start using Fabric Databases. View on-demand sessions with database experts and the Microsoft product team to learn just how easy it is to get started. Watch now
Does anybody know enough of AWS to tell me if I can connect a DEV AWS SQL Server instance to the same gateway cluster as the PROD AWS SQL Sever?
There's a legit business need. For a very high priority project we pointed the AWS DEV SQL server job to source from a production system so we can modify our sql jobs and have those results available in production reporting.
IT is telling me I can't have the 2 together. That I will get a gateway installed on a dev server that connects to the AWS DEV SQL Server AND a Prod Databricks repository. But my gateway went down with dev this last weekend and can't be relied on for production reports. I honestly had never heard of a dev gateway for production env.. but I was told this was dictated by the AWS Dev sql source.
They also offered me gateway cluster installed on Prod Sever with connetions to Prod AWS SQL server and Prod databricks connection. I can't move to this until the work is done in dev and makes it to production as they aren't offering to install the DEV AWS SQL server connection in same cluster. The Dev connect is the thowaway solutoin at a point.. lol.. but I can assure you we're not at that point. the key is that it's a production report that needs production uptime and until we work out our coding around crazy calendaring items.
So the question hinges on IT telling me there's no way around having a gateway cluster that has AWS Dev SQL sever installed on anything but a dev server and that the 2 can't be in the same production gateway cluster because they're on different servers?
My spidy sense tells me there's absolutely something that could be done in AWS on the backend that could make the DEV repository and table I need available in the AWS Prod instance? no? a crafty permissions thing, a grant select on the dev table if we can create a backend connection for them?
Does anybody have any presciptive steps to take in AWS to pull this off?
I can confirm this is the issue. Is there a solutoin prior to the gateway cluster? Entirely based in AWS. I recall at a previous employer using Glue and getting help opening up aws connectivity beween enviorments to move large amounts of test data (in s3) where we needed it. If all we're talking about is 1 table of data is there a way to load it to prod from the other so I don't need gateway cluster to reach both but rather an AWS-blessed way to make it happen on the backend?
This seems the way to do it: What is AWS Data Pipeline? - AWS Data Pipeline (amazon.com)
a gateway is brokering between the data source and the Azure service. It needs to "see" both. So this becomes more of a network design/compartmentalization question. If your gateway cluster member can "see" both AWS SQL Server instances then you're fine. But if your IT team did the right thing and firewall the environments then you most likely can't do it.
March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!
Your insights matter. That’s why we created a quick survey to learn about your experience finding answers to technical questions.
Arun Ulag shares exciting details about the Microsoft Fabric Conference 2025, which will be held in Las Vegas, NV.
User | Count |
---|---|
129 | |
90 | |
75 | |
58 | |
53 |
User | Count |
---|---|
200 | |
104 | |
101 | |
67 | |
55 |