topic Migration from On-Prem File Share to Azure Blob Issues in Developer

Migration from On-Prem File Share to Azure Blob Issues

Anonymous — Mon, 27 Nov 2017 17:23:47 GMT

Scenario:

Customer's Power BI data model is built using text files and excel spreadsheets that are stored on an on-premise file share using a combination of 'Text/CSV' and 'Folder' data sources. They also use a 'SQL Server Database' data source that is also stored on-prem. This model has been published to the PBI Service and both on-demand and scheduled refreshes occur as expected and complete successfully using the customers on-prem gateway server.

Customer now wants to migrate all of their text and excel files to Azure Blob storage while leaving the SQL database on-prem. The expectation is that the Power BI Service will access the text and excel files via the Azure back end and still use the gateway to access the on-prem SQL database.

Issue:

Using Power BI Desktop, the data sources are changed as needed to point to the Azure Blob storage location. Two different migration methods were tested:

Create a new data source and change existing queries to reference the new data source in the 'Advanced Editor' view of the existing queries.
Change existing query to point to Azure Blob Storage in Advanced Editor.

With both approaches, the data source change was successful in Power BI Desktop. The files were pulled from Azure and the model worked properly.

In both approaches, however, once published to the Power BI Service, the data set would no longer refresh.

Symptoms:

When the newly published data set was accessed in the Power BI service interface and the 'Refresh Now' option was chosen, the following pop-up occurs:

Install the data gateway (personal mode) to refresh. etc.

If the 'Schedule Refresh' option is chosen, the same notification appears at the top of the popup and the option to choose the on-prem gateway is now greyed out:

You don't have any gateway installed or configured for the data sources in this dataset. Please install....etc.

Expected Behavior:

Expectation is that Power BI will pull the data stored in the Azure Blob storage via the Azure backend and use the on-prem gateway to access the on-prem SQL Server datasource.

Re: Migration from On-Prem File Share to Azure Blob Issues

Eric_Zhang — Tue, 28 Nov 2017 08:03:18 GMT

Anonymous wrote:

Scenario:

Customer's Power BI data model is built using text files and excel spreadsheets that are stored on an on-premise file share using a combination of 'Text/CSV' and 'Folder' data sources. They also use a 'SQL Server Database' data source that is also stored on-prem. This model has been published to the PBI Service and both on-demand and scheduled refreshes occur as expected and complete successfully using the customers on-prem gateway server.

Customer now wants to migrate all of their text and excel files to Azure Blob storage while leaving the SQL database on-prem. The expectation is that the Power BI Service will access the text and excel files via the Azure back end and still use the gateway to access the on-prem SQL database.

Issue:

Using Power BI Desktop, the data sources are changed as needed to point to the Azure Blob storage location. Two different migration methods were tested:

Create a new data source and change existing queries to reference the new data source in the 'Advanced Editor' view of the existing queries.

Change existing query to point to Azure Blob Storage in Advanced Editor.

Anonymous

I don't have any problem when refreshing a dataset importing data from Azure Blob storage. It doesn't need any gateway. As to your migartion methods, I don't know much details, however I'd suggest you follow below steps.

"Get-data" from the connector "Azure Blob storage".
Delete the csv table from on-premise file and rename the table from step 1 with the deleted table's name.
After all "Delete and Rename" from step2, configure the credential for the Azure blob storage in the dataset refresh page.

Re: Migration from On-Prem File Share to Azure Blob Issues

Anonymous — Tue, 28 Nov 2017 15:50:41 GMT

Remember that I am talking about a hybrid scenario. I still need the gateway to access the SQL table that is my date table. If I create the data source from scratch as existing in Azure, I dont have an issue. This also may be related to the migration of a folder data source to an Azure Blob container datasource, not sure.

The problem seems to be with cached information with the gateway, either in the project or in the service itself. It appears as if it is still trying to use the gateway to access the Azure blob data source.

In your repro, did you first connect through the gateway to that data source and the change to Azure? For some reason, I run into a lot of problems