This guide will cover how to create a OneLake shortcut to a VPC-protected Google Cloud Storage (GCS) bucket.
Why use the on-premises-data gateway?
Today, organizations are protecting data by leveraging network security capabilities like virtual networks, firewalls and virtual protected clouds (VPC). To access data securely and to provide a bridge between protected environments and Microsoft Fabric, an on-premises data gateway can be used.
Although the name might suggest that the on-premises data gateway can only be used to access your data that is on-premises, it can actually be used to access any data that is protected by any type of firewall or virtual network, including the virtual protected clouds on Google Cloud Platform (GCP). For more information about the on-premises data gateway refer to our documentation.
Setting up a gateway is an easy process. You need to provision a Compute Engine virtual machine (VM) instance within your virtual private cloud; and configure (or open) appropriate ports to securely communicate with Microsoft Fabric. In this tutorial, we will walk you through the steps to complete end-to-end setup.
If you are already using the on-premises data gateway within Fabric for other items, like Pipelines, dataflows or Power BI, you can use the same instance of on-premises data gateway, as long as it also has access to your GCS bucket inside the VPC as shown in the diagram below.
At a high-level, the setup process consists of the following steps:
- Create a public subnet in your VPC environment and assign security groups to the GCS bucket subnet.
- Create a Compute Engine VM instance within the second subnet.
- Install the on-premises data gateway on the Compute Engine VM instance.
- Open the right ports to Fabric service.
- Create a shortcut using the on-premises data gateway.
Prerequisites
- A Virtual Private Cloud (VPC) provisioned in GCP, follow this guide to create a VPC.
- An GSC bucket that is only accessible from within the GCP VPC.
Step-by-step set up
1. Create a public subnet in your VPC environment and assign NSG’s.
- If you don’t have a public subnet follow this guide to create an internet gateway for a subnet in your VPC.
2. Create a Compute Engine VM instance within the public subnet.
- Install a VM instance running the Windows operating system by following these steps. The minimum installation requirements for the machine SKU and OS version.
- Launch Compute Engine VM instance in the subnet of your VPC.
3. Install the on-premises data gateway on the VM instance
- Connect to your VM instance using a Remote Desktop Protocol (RDP) client, following this guide to learn how to connect over RDP, you will need your private key file from the previous step.
- After signing in to the VM instance, follow the steps to install the standard OPDG client.
4. Open the right ports to the Fabric service
- If a firewall blocks outbound connections, configure the firewall to allow outbound connections from the gateway to its associated Azure region. The firewall rules on the gateway need to be updated to allow outbound traffic from the gateway server to the following endpoints.
5. Create a shortcut to GCS
- Follow the guide to create a Google Cloud Storage shortcut. Select the OPDG created in the previous steps to ensure the connection goes through the VPC.
For more information on troubleshooting guidance on the on-premises data gateway, refer to the troubleshooting documentation.