Forum Discussion
Helper IConfigure Power BI Data Gateway to Data Lake Gen 2
With some help from colleagues we made this work today.
1) So the first thing was the data lake firewall settings.
I had set the private vnet data lake firewall to allow access from selected networks.
My vnet firewall allows host to host connection so Azure Storage explorer on my Gateway VM connects to the Data Lake fine. But that is not enough. On the data lake firewall make sure you have added the subnet ( your private vnet) and not left this blank. Should look something like this.2) My gateway installed with a local servce account. For now I left this alone, but believe a service account and spn registration are the way forward to support delegation.
3) Adding a data source in the Power BI Manage Gateways, brings up the screen in your post.
Server is the dfs URL for the container: https://<storage_account>.dfs.core.windows.net
Full Path is the resource part of the URI
https://<storage_account>.dfs.core.windows.net/<path>My container name is /test and leaving it at that brings back a list of files at the get data stage.
For Authentication I used the key option and copied the access key from the azure portal: Storage Account-> Access Keys (Not SAS Tokens)At this point my first sucessful save worked.
Click on your cluster name and it expands to list your data sources and "test all" should report sucess above each form.
4) In Power BI and your workspace add New and choose Data Flow.
Choose Add New Entries
Scroll or search data sources and select data lake gen 2
This form appears and it does not work too well.
The url is the https://<>..dfs.core.windows.net/<container> // Copy from Azure Storage explorer
The drop down lists your gateway.
And the authentication throws lots of strange errors, and despite that works if you click on Next.
I think this is all still in preview so good luck with support.
Regards
Keith
With some help from colleagues we made this work today.
1) So the first thing was the data lake firewall settings.
I had set the private vnet data lake firewall to allow access from selected networks.
My vnet firewall allows host to host connection so Azure Storage explorer on my Gateway VM connects to the Data Lake fine. But that is not enough. On the data lake firewall make sure you have added the subnet ( your private vnet) and not left this blank. Should look something like this.
2) My gateway installed with a local servce account. For now I left this alone, but believe a service account and spn registration are the way forward to support delegation.
3) Adding a data source in the Power BI Manage Gateways, brings up the screen in your post.
Server is the dfs URL for the container: https://<storage_account>.dfs.core.windows.net
Full Path is the resource part of the URI https://<storage_account>.dfs.core.windows.net/<path>
My container name is /test and leaving it at that brings back a list of files at the get data stage.
For Authentication I used the key option and copied the access key from the azure portal: Storage Account-> Access Keys (Not SAS Tokens)
At this point my first sucessful save worked.
Click on your cluster name and it expands to list your data sources and "test all" should report sucess above each form.
4) In Power BI and your workspace add New and choose Data Flow.
Choose Add New Entries
Scroll or search data sources and select data lake gen 2
This form appears and it does not work too well.
The url is the https://<>..dfs.core.windows.net/<container> // Copy from Azure Storage explorer
The drop down lists your gateway.
And the authentication throws lots of strange errors, and despite that works if you click on Next.
I think this is all still in preview so good luck with support.
Regards
Keith
tracyng0905Just one more thing to note:
Creating a DataLake on a VNET will automatically add a private end point for <>.blob.core.windows.net
But it does not create a private end point for <>.dfs.core.windows.net and that will remain a public service end point.
To fix that issue in the portal select the storage account you created and added to the VNET, and select "private end point connections" and add a new private end point for DFS. Make sure that you select the same region as your data lake, or the subnet drop down will not find anything useful. And choose the resource and sub resource as your data lake and "dfs" respectively. That will move the DFS URL to your private VNET.
Does it work with pwbi pro or pwbi premium (not by capacity) ???
PowerBI and Power BI premium work the same with this pattern.
