Power BI Dataflow to Azure Data Lake Gen2 Setup

[b5lurker]

Excited to see all of the new Power BI Dataflow capabilities especially with using Azure Data Lake Gen2 as the storage location for those Dataflows.

But, I have been trying to get it setup using the Microsoft provided step-by-step guide on: https://docs.microsoft.com/en-us/power-bi/service-dataflows-connect-azure-data-lake-storage-gen2. The specific steps that I have been unable to complete are in the 'Grant Power BI permissions to the file system' section steps #7 and #8. I have only tried it using Azure Storage Explorer v1.6 since that is the method shown in the documentation.

When I try to Add the Object IDs for the Power BI Service and Power Query Online as shown in the screenshots I only get an error saying that it cannot find those User IDs/Groups. I have attached the full error message captured in ASE to see if that might help track it down. I did this with my elevated permissions Azure account and also had our admin grant me full Admin rights over our tenant and could not get it working.

Has anyone been able to complete the steps to grant the Power BI Service and Power Query Online applications access to the powerbi blob container in their Azure Data Lake Store Gen2? I even re-created the ADLSg2 yesterday since I had it created in the original private preview. Without these steps I have not been able to complete the process in the Power BI Admin screen to connect Power BI up to the Data Lake for Dataflows.

Thanks,

Steve

Below is the error message from ASE when trying to Add the Power BI Service Object ID in the Manage Access dialog:

FROM CHILD PROCESS 24608: {"id":"MessagePassingHostProxy151","messageType":"FunctionResponse","response":{"type":"error","error":"{\n  \"message\": \"{\\\"message\\\":\\\"HTTP ERROR 404: Not Found\\\",\\\"response\\\":{\\\"statusCode\\\":404,\\\"body\\\":\\\"{\\\\\\\"odata.error\\\\\\\":{\\\\\\\"code\\\\\\\":\\\\\\\"Request_ResourceNotFound\\\\\\\",\\\\\\\"message\\\\\\\":{\\\\\\\"lang\\\\\\\":\\\\\\\"en\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"Resource 'ee4bcd73-1e25-4154-befe-f8180e3f14d5' does not exist or one of its queried reference-property objects are not present.\\\\\\\"},\\\\\\\"requestId\\\\\\\":\\\\\\\"724657f5-7d0c-4896-9230-5226b96ff9b6\\\\\\\",\\\\\\\"date\\\\\\\":\\\\\\\"2018-12-11T16:34:51\\\\\\\"}}\\\",\\\"headers\\\":{\\\"cache-control\\\":\\\"no-cache\\\",\\\"pragma\\\":\\\"no-cache\\\",\\\"content-type\\\":\\\"application/json; odata=minimalmetadata; streaming=true; charset=utf-8\\\",\\\"expires\\\":\\\"-1\\\",\\\"server\\\":\\\"Microsoft-IIS/10.0\\\",\\\"ocp-aad-diagnostics-server-name\\\":\\\"QAcXmVYUoJaUSrw2ftJmfZfMjTyUjtSkz5dJvbTYsQ0=\\\",\\\"request-id\\\":\\\"724657f5-7d0c-4896-9230-5226b96ff9b6\\\",\\\"client-request-id\\\":\\\"587c72a6-407f-4024-95b6-fbc710401123\\\",\\\"x-ms-dirapi-data-contract-version\\\":\\\"1.6\\\",\\\"ocp-aad-session-key\\\":\\\"hqhNfIg59I6k-OSo-mWlGPFJUAeZDoSngtOC2dH4v4szDJot9OmJGoysNRoVpgZKj_T_ctHrILdmnh-x_E_1VTEIkhcExHCXJk-BPGZKyTzxWQqqAJYh7r3biDvlbfg1.tMDQP1ZfH1teuhHoVGVcULDVzDSD4PiGem3E2On_fFM\\\",\\\"dataserviceversion\\\":\\\"3.0;\\\",\\\"strict-transport-security\\\":\\\"max-age=31536000; includeSubDomains\\\",\\\"access-control-allow-origin\\\":\\\"*\\\",\\\"x-aspnet-version\\\":\\\"4.0.30319\\\",\\\"x-powered-by\\\":\\\"ASP.NET\\\",\\\"duration\\\":\\\"430184\\\",\\\"date\\\":\\\"Tue, 11 Dec 2018 16:34:51 GMT\\\",\\\"connection\\\":\\\"close\\\",\\\"content-length\\\":\\\"294\\\"},\\\"request\\\":{\\\"uri\\\":{\\\"protocol\\\":\\\"https:\\\",\\\"slashes\\\":true,\\\"auth\\\":null,\\\"host\\\":\\\"graph.windows.net\\\",\\\"port\\\":443,\\\"hostname\\\":\\\"graph.windows.net\\\",\\\"hash\\\":null,\\\"search\\\":\\\"?api-version=1.6\\\",\\\"query\\\":\\\"api-version=1.6\\\",\\\"pathname\\\":\\\"/cb2bab3d-7d90-44ea-9e31-531011b1213d/users/ee4bcd73-1e25-4154-befe-f8180e3f14d5\\\",\\\"path\\\":\\\"/cb2bab3d-7d90-44ea-9e31-531011b1213d/users/ee4bcd73-1e25-4154-befe-f8180e3f14d5?api-version=1.6\\\",\\\"href\\\":\\\"https://graph.windows.net/cb2bab3d-7d90-44ea-9e31-531011b1213d/users/ee4bcd73-1e25-4154-befe-f8180e3f14d5?api-version=1.6\\\"},\\\"method\\\":\\\"get\\\",\\\"headers\\\":{\\\"Content-Type\\\":\\\"application/json\\\",\\\"Authorization\\\":\\\"Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IndVTG1ZZnNxZFF1V3RWXy1oeFZ0REpKWk00USIsImtpZCI6IndVTG1ZZnNxZFF1V3RWXy1oeFZ0REpKWk00USJ9.eyJhdWQiOiJodHRwczovL2dyYXBoLndpbmRvd3MubmV0LyIsImlzcyI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0L2NiMmJhYjNkLTdkOTAtNDRlYS05ZTMxLTUzMTAxMWIxMjEzZC8iLCJpYXQiOjE1NDQ1NDU3OTEsIm5iZiI6MTU0NDU0NTc5MSwiZXhwIjoxNTQ0NTQ5NjkxLCJhY3IiOiIxIiwiYWlvIjoiQVZRQXEvOEpBQUFBTExXL2dQcVV2ZmhjODM1MGtrYXhKdVFsMGpUcU00N3dwTG14WGo3b05oZGJHRTRYTjZXdk01VFJYanhuWm9jdzh2Z2I3emZDWXR6ZWw5NFo4WHRsd0RKZ3l0LzFqSlFWbWZzNGlVZm44STQ9IiwiYW1yIjpbInB3ZCIsIm1mYSJdLCJhcHBpZCI6IjA0YjA3Nzk1LThkZGItNDYxYS1iYmVlLTAyZjllMWJmN2I0NiIsImFwcGlkYWNyIjoiMCIsImZhbWlseV9uYW1lIjoiV2FrZSIsImdpdmVuX25hbWUiOiIhU3RldmUiLCJpcGFkZHIiOiIyMDYuODAuMjA5LjE5NiIsIm5hbWUiOiIhU3RldmUgV2FrZSIsIm9pZCI6ImZjOTAxZTNkLWE0MGQtNDgzOC05MDZlLWE5Y2UwYTk0ZTk0MyIsIm9ucHJlbV9zaWQiOiJTLTEtNS0yMS0yMTQ3MjQyNzI2LTM2MzMxNTgwLTE4ODQ0MTQ0NC0xNzI1NTkiLCJwdWlkIjoiMTAwMzdGRkVBODFDNTIyNyIsInNjcCI6IjYyZTkwMzk0LTY5ZjUtNDIzNy05MTkwLTAxMjE3NzE0NWUxMCIsInN1YiI6Ilo0TjlINTQ4alBPNUlGSVE0dS1FU01OdllHRE02WndNVEs3dm80LVljSzQiLCJ0ZW5hbnRfcmVnaW9uX3Njb3BlIjoiTkEiLCJ0aWQiOiJjYjJiYWIzZC03ZDkwLTQ0ZWEtOWUzMS01MzEwMTFiMTIxM2QiLCJ1bmlxdWVfbmFtZSI6IiFzd2FrZUBicnduY2FsZC5jb20iLCJ1cG4iOiIhc3dha2VAYnJ3bmNhbGQuY29tIiwidXRpIjoiOGVCcThEa0o4VVNrT05RQ2dZa2tBQSIsInZlciI6IjEuMCJ9.DVQ1-dpeAfEPX9Lf62JuVtheaAiTNECZpTfoihlL93TKFzEJW-N8alEwDLAfkGTwvj92GRgi7MThS-HqWGfAla4C0ntKxuzq8NhTeGEuACNXVUDNO5mtfYi1W2jJDJirduMSOgdNkkBqCnFvVMqM49cUAYEaPOD-Wn-Fb-U_L3K-hOaZ2EL2_aFLXpm5w-JJs9MPQXcoOSR1rT1x5L9x7At0hIn-A97RzzabFTAMfoolAVDymjLrT2It5jf1qn1vZ1aGXBZPLH_uF7_Aq_v_gTPECuuj-B2rJV3FokQHHCYu9iNFl7lt95ZP_lv_Q9nH01QNp-xjxyHfTrub7iCYCA\\\",\\\"content-length\\\":0}}},\\\"body\\\":\\\"{\\\\\\\"odata.error\\\\\\\":{\\\\\\\"code\\\\\\\":\\\\\\\"Request_ResourceNotFound\\\\\\\",\\\\\\\"message\\\\\\\":{\\\\\\\"lang\\\\\\\":\\\\\\\"en\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"Resource 'ee4bcd73-1e25-4154-befe-f8180e3f14d5' does not exist or one of its queried reference-property objects are not present.\\\\\\\"},\\\\\\\"requestId\\\\\\\":\\\\\\\"724657f5-7d0c-4896-9230-5226b96ff9b6\\\\\\\",\\\\\\\"date\\\\\\\":\\\\\\\"2018-12-11T16:34:51\\\\\\\"}}\\\"}\"\n}"}}

[yehong]

Azure Storage Explorer 1.6.2 has been officially released https://docs.microsoft.com/en-us/azure/vs-azure-tools-storage-explorer-relnotes#version-162

Issues with ASE 1.6.1 are fixed with 1.6.2, please let us know of any issues you encounter with it during setup of storage with PowerBI as outlined here https://docs.microsoft.com/en-us/power-bi/service-dataflows-connect-azure-data-lake-storage-gen2

[Picci]

Hi @yehong, what about changing the ADLS Gen2 account to be associated with Power BI?

What about if I need to move my storage account to another subscription?

Thanks,

Elisa

[yehong]

You can move the storage account between subscriptions and resource-groups as long as it's kept under the same tenant.

[Picci]

@yehong thanks for your reply, is it possible to do it from the Power BI Service Admin Portal? At the moment I'm not able to change what I specified in Dataflow storage (preview) under https://app.powerbi.com/admin-portal/dataflows. Thanks again, Elisa

[yehong]

At the moment we're not caching the subscription id and resource-group name therefore there is no action you should take in Power BI portal. Please note that the storage should be kept under the same tenant.

[Picci]

Ok, just to see if I've understood correctly: I can move the ADLS2 storage to another subscription and resource group, as far as: - they are under the SAME tenant - the ADLS2 storage name remains the SAME In the future it will be possible to unlink/link storage accounts? Or to have more than one storage account to manage (and then decide from time to time which storage account to link to which Power BI Workspace)?

[yehong]

You understood correctly.

We're always trying to improve the experience we deliver:

1. Can share why you're looking to move a storage to a different subscription / resource-group?

2. Link/Unlink storage accounts - can you please create a new idea in Power BI community? explaining why do you need this functionality / how you expect it to behave?

3. Multiple storage accounts - can you please create a new (separate) idea in Power BI community? explaining why do you need this functionality / how you expect it to behave?

You can add new Power BI Ideas here: https://ideas.powerbi.com/forums/265200-power-bi-ideas

Please share links to these ideas once created so others can add their voice as well !

[Picci]

Hi @yehong,

I'm looking to move the ADLS2 used by Power BI Dataflows to a different subscription / resource-group because at the beginning I used for testing purpose my Visual Studio MPN Subscription, but now I want to move the ADLS2 to a pay-as-you-go subscription within the same tenant.

As you suggested, I've just created two new Power BI ideas

https://ideas.powerbi.com/forums/265200-power-bi-ideas/suggestions/36495385-link-unlink-azure-data-l...

https://ideas.powerbi.com/forums/265200-power-bi-ideas/suggestions/36495421-support-multiple-azure-d...

...waiting for your comments, suggestions and hopefully votes! 😉

Elisa

[markusth]

I surely checked several times that I followed all steps to connect my ADLS gen 2 to Power BI, but I still get no success. 

1. I made sure the ADLS is in the same location then my PBI tenant (in my case North Europe)
2. I give the PBI Service access with the reader role
3. I give PBI Svc and Power Query full access to the container folder "powerbi"
4. I even tried full access to all 

any suggestion why I still get no success?

Aktivitäts-ID14edddab-cb86-48c7-9ce4-6ffc3e48c1ac

Anforderungs-ID7eb3a30d-892c-c8fc-a5ba-018643494695

Korrelations-IDac823118-22e0-3bc8-a0fb-60d41fde1b62

ZeitFri Dec 21 2018 12:54:24 GMT+0100 (Mitteleuropäische Normalzeit)

Version13.0.7683.188

Cluster-URIhttps://wabi-north-europe-redirect.analysis.windows.net

[julioleite]

I am with the same issue. Checked the points several times and got the error.

[sagivh]

Hi everyone, we acknowledge the reported setup issues. we have found a potential issue in Azure Storage Explorer and are working on a mitigation. 

We will update this post when we have more information and ETA. 

Thank you. 

[RyanTheDataBus]

Hi,

I'm having the same issue.  Looking forward to any help you can provide.

Cheers,

Ryan

Something went wrong
Error while connecting your Data Lake Storage account
Please try again later or contact support. If you contact support, please provide these details.
Activity ID: 20b9daf2-da0e-4dc0-a266-b8aecd433a94
Request ID: c86756c0-210b-13ec-23a6-a1ea7d6de12e
Correlation ID: afa24dbe-d215-d05f-12e4-ae9ae7fa3c46
Time: Fri Dec 28 2018 13:27:05 GMT-0800 (Pacific Standard Time)
Version: 13.0.7683.188
Cluster URI: https://wabi-us-north-central-b-redirect.analysis.windows.net

[ruiromano]

Hi,

Same issue here 😞

Error:

Activity ID7ae90109-cd7f-4caf-a342-0a0c1700a3b3
Request ID9385312a-6d0c-133b-4530-72f88ad0bbbb
Correlation IDd36f7917-72d5-f8dc-5567-7a220a6c640f
TimeSat Dec 29 2018 17:21:23 GMT+0000 (Western European Standard Time)
Version13.0.7683.237
Cluster URIhttps://wabi-west-europe-redirect.analysis.windows.net

[Michael7 · Accepted solution]

Hi,
 
There is a temporary issue with settings ACLs through Microsoft Azure Storage Explorer, until this is resolved, as a temporary mitigation you can try using this powershel script:https://setacls.blob.core.windows.net/scriptfolder/setacls.ps1
 
In order to run the script, your global tenant admin needs to have "Storage Blob Data Contributor (Preview)" role on your storage account.
 
Instructions on how to run this script:
    
1. Login as global tenant administrator into Azure portal and open PowerShell (Cloud Shell) in your browser
2. Choose Azure CLI 2.0 (type ‘az’)
3. Upload the script (there is a button for it)
4. Run the script. For example:  /home/admin/setacls -storageaccountname your_alds_gen2_storage_account_name

The script will ask you to sign in, please follow the instructions.
 
Thanks,
Michael

Hi,

There is no more need in this workaround, please use ASE 1.6.2 which is availably for update and download.

Thanks,

Michael

[BhaveshPatel]

Hi Micheal,

I managed to run the provided powershell script and the set up ran successfully. Set ACLs App has been successfully created and have Azure Storage Access on my data lake storage account as well as Read Permissions on the Azure Active Directory. I have added the objectId of this app to the powerbi folder on my storage account with full access.

There is still an issue with connecting Dataflows to the Azure Data Lake Gen2 account.

Error Message:

Something went wrong

Error while connecting your Data Lake Storage account
Please try again later or contact support. If you contact support, please provide these details.

Activity IDbd89f749-7e02-4af5-9f24-8de3dd9f3b42

Request IDf41b103f-e7fc-4694-a396-f82380a2fd90

Correlation ID1af2a28f-91c3-565b-346b-0bfca13e556c

TimeFri Jan 04 2019 11:21:04 GMT+1100 (AUS Eastern Summer Time)

Version13.0.7683.237

Cluster URIhttps://wabi-australia-east-a-primary-redirect.analysis.windows.net

Can you please advise the solution for this.

Thanks

Bhavesh

[b5lurker]

I can confirm that I was able to run these steps and get the Dataflows setup with our ADLSg2 Data Lake in Azure! 🙂

Thanks, now I can start to play with this new feature! 🙂

Steve

[Anonymous]

Hi all,

Thanks! I can also confirm that this works for me! Great work!

[BhaveshPatel]

Hi @Anonymous,

Can you please briefly explain what are the series of steps performed to make this work successfully.

It would be really appreciated! I am trying all the recommended steps as explained by @Michael7 but have no success.

This would really assist me in debugging the process.

Thanks,

[BhaveshPatel]

Hi all,

The problem is resolved after updating the azure storage explorer and adding Power BI service as a reader role on the data lake storage account.

Thank you all for the wonderful support,

Thanks

Bhavesh

[orashare92]

Hi I have the same problem. The resolution doesn't work for me. I still got the error: Something went wrong Error while connecting your Data Lake Storage account Please try again later or contact support. If you contact support, please provide these details. Activity ID: 76cd1c0f-ee44-4a80-a659-054ae87c5d51 Request ID: cafe05ad-a085-78a8-0fa1-7e19c61dbb8e Correlation ID: 7497d11e-8a5c-06cf-5a97-ccb11a97dd31 Time: Mon Jan 07 2019 08:50:19 GMT+0100 (Środkowoeuropejski czas stand.) Version: 13.0.7840.225 Cluster URI: https://wabi-europe-north-b-redirect.analysis.windows.net Regards Maciek