Forum Discussion

mikeeq's avatar
mikeeq
Frequent Visitor
2 years ago

Data Gateway PowerShell module issue with `Add-DataGatewayClusterMember` command `-RegionKey` param

Hi all,

I recently encountered an issue with the latest version of DataGateway PowerShell module (latest version of PS7 and DataGateway service): https://www.powershellgallery.com/packages/DataGateway/3000.196.145

 

 

 

PS C:\> (Get-Module DataGateway).version

Major Minor Build Revision
----- ----- ----- --------
3000 196 145 -1

PS C:\> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.4.3

PS C:\> (Get-Item "C:\Program Files\On-premises data gateway\Microsoft.PowerBI.EnterpriseGateway.exe").VersionInfo.FileVersion
3000.226.5

 

 

 

 

 

`Add-DataGatewayClusterMember` command doesn't seem to respect `-RegionKey` parameter, whenever it's passed to the command, it cannot find the GatewayClusterId in the specified region. 

 

 

 

# Info: Gateway cluster xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx is created in uksouth region
PS C:\> Add-DataGatewayClusterMember -GatewayClusterId xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx -GatewayName datagatewaypoc-2 -RecoveryKey $secureRecoveryKey -OverwriteExistingGateway -RegionKey uksouth

Add-DataGatewayClusterMember: Gateway cluster xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx was not found

 

 

 

 

 

When the cluster is deployed in a default PowerBI tenant region and when `-RegionKey` parameter is not passed then it works as it should.

 

 

 

# Info: Gateway cluster xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx is created in PowerBI default tenant (northeurope in that case)
PS C:\> Add-DataGatewayClusterMember -GatewayClusterId xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx -GatewayName datagatewaypoc-2 -RecoveryKey $secureRecoveryKey -OverwriteExistingGateway


GatewayId GatewayObjectId GatewayName GatewayType
--------- --------------- ----------- -----------
xxxxxx xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx datagatewaypoc-2 Resource

 

 

 

 


Also, `Add-DataGatewayClusterMember` is not documented at all, the only mention is in the blog post regarding release notes from 2023.10:
https://learn.microsoft.com/en-us/powershell/module/datagateway/remove-datagatewayclustermember?view=datagateway-ps

10 Replies

  • mikeeq's avatar
    mikeeq
    Frequent Visitor

    Seems like API requests are missing "&region=westeurope" parameter at the end of URI, or region specific API endpoint is not filtering data (gateways in that case) per region.

     

     

     

    PS C:\Users\vagrant> add-datagatewaycluster -RecoveryKey $secureRecoveryKey -Name DataGatewayPoC -OverwriteExistingGateway -RegionKey westeurope -Verbose -Debug
    DEBUG: 12:18:57 - Add-DataGatewayCluster begin processing with ParameterSet __AllParameterSets.
    DEBUG: 12:18:57 - Cmdlet version: 3000.196.145
    VERBOSE: Request Uri: https://api.powerbi.com/v2.0/myorg/me/gatewayRegions
    VERBOSE: Status Code: OK (200)
    VERBOSE: Request Uri: https://wabi-west-europe-redirect.analysis.windows.net/v2.0/myorg/me/gatewayClusters?$expand=permissions,memberGateways
    VERBOSE: Status Code: OK (200)
    VERBOSE: Request Uri: https://wabi-west-europe-redirect.analysis.windows.net/unifiedgateway/gateways/CreateGatewayWithApps
    VERBOSE: Status Code: OK (200)
    DEBUG: Microsoft.PowerBI.DataMovement.ExternalContracts.API.CreateGatewayResponse
    
    DEBUG: 12:19:02 - Add-DataGatewayCluster end processing.
    GatewayId GatewayObjectId                      GatewayName    GatewayType
    --------- ---------------                      -----------    -----------
    xxxxxxx   xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx DataGatewayPoC Resource

     

     

     

     

    PS C:\> Add-DataGatewayClusterMember -RegionKey westeurope -GatewayName "DataGatewayPoC-2" -RecoveryKey $secureRecoveryKey -OverwriteExistingGateway -Debug -Verbose -GatewayClusterId "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
    DEBUG: 12:19:51 - Add-DataGatewayClusterMember begin processing with ParameterSet __AllParameterSets.
    DEBUG: 12:19:51 - Cmdlet version: 3000.196.145
    VERBOSE: Request Uri: https://api.powerbi.com/v2.0/myorg/me/gatewayRegions
    VERBOSE: Status Code: OK (200)
    VERBOSE: Request Uri: https://wabi-west-europe-redirect.analysis.windows.net/v2.0/myorg/me/gatewayClusters?$expand=permissions,memberGateways
    VERBOSE: Status Code: OK (200)
    Add-DataGatewayClusterMember: Gateway cluster xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx was not found
    DEBUG: System.ArgumentException: Gateway cluster xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx was not found
       at Microsoft.DataMovement.Powershell.Commands.DataGateway.AddDataGatewayClusterMember.ExecuteCmdletAsync()
       at Microsoft.DataMovement.Powershell.Commands.Common.PowerBICmdlet.ProcessRecordAsync()
    DEBUG: 12:19:52 - Add-DataGatewayClusterMember end processing.

     

     

     

     

    PS C:\Users\vagrant> get-dataGatewayCluster -RegionKey westeurope -Verbose -Debug
    DEBUG: 12:29:37 - Get-DataGatewayCluster begin processing with ParameterSet __AllParameterSets.
    DEBUG: 12:29:37 - Cmdlet version: 3000.196.145
    VERBOSE: Request Uri: https://api.powerbi.com/v2.0/myorg/me/gatewayClusters?$expand=permissions,memberGateways&region=westeurope
    VERBOSE: Status Code: OK (200)
    
    Id                    : xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    Name                  : DataGatewayPoC
    Description           :
    Permissions           : {app-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx, app-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx,
                            xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}
    Type                  : Resource
    MemberGateways        : {DataGatewayPoC}
    Datasources           : {}
    LoadBalancingSettings :
    AllowableOptions      : Microsoft.PowerBI.ServiceContracts.Api.ODataDictionary`1[System.Boolean]
    StaticCapabilities    : 0
    MaxPermission         : Microsoft.PowerBI.ServiceContracts.Api.Permission
    
    DEBUG: 12:29:38 - Get-DataGatewayCluster end processing.

     

     

     

    • hackcrr's avatar
      hackcrr
      Memorable Member

      Hi, mikeeq 

      Debug logs indicate that the Add-DataGatewayClusterMember command does not region to include this parameter in its API request URI, while the Get-DataGatewayCluster command does.
      RegionKey As a temporary workaround, you might consider deploying the gateway in a default region that doesn't require the parameter, as you mentioned, it works fine in this case. Since this Get-DataGatewayCluster command region correctly includes the parameter, you can use it to verify the gateway cluster and its region. Then, try adding cluster members without specifying a region.
      This is a potential scripted workaround:

      # Get Gateway Cluster Details with RegionKey
      $gatewayClusterId = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
      $gatewayCluster = Get-DataGatewayCluster -RegionKey "westeurope" -Verbose -Debug
      
      if ($gatewayCluster) {
          Write-Output "Gateway cluster found: $($gatewayCluster.Name) in region $($gatewayCluster.Region)"
      } else {
          Write-Output "Gateway cluster not found."
      }
      
      # Attempt to add a cluster member without RegionKey
      try {
          Add-DataGatewayClusterMember -GatewayClusterId $gatewayClusterId -GatewayName "DataGatewayPoC-2" -RecoveryKey $secureRecoveryKey -OverwriteExistingGateway -Verbose -Debug
      } catch {
          Write-Output "Error encountered: $_"
          # Log error details for further analysis
      }

      If this workaround doesn't work or you don't want to use it, you can just report the error. Below is a sample template you can use to report a problem:

      Subject: Bug Report: Add-DataGatewayClusterMember Command Ignoring -RegionKey Parameter in DataGateway Module

      Description:
      I am encountering an issue with the Add-DataGatewayClusterMember command in the latest DataGateway PowerShell module (version 3000.196.145). When using the -RegionKey parameter, the command fails to find the specified GatewayClusterId in the region. The -RegionKey parameter works correctly with other commands, such as Get-DataGatewayCluster.

      Steps to Reproduce:
      Create a Gateway Cluster in the westeurope region.
      Attempt to add a cluster member using the Add-DataGatewayClusterMember command with the -RegionKey parameter.

      # Add Gateway Cluster
      Add-DataGatewayCluster -RecoveryKey $secureRecoveryKey -Name DataGatewayPoC -OverwriteExistingGateway -RegionKey westeurope -Verbose -Debug
      
      # Add Cluster Member
      Add-DataGatewayClusterMember -RegionKey westeurope -GatewayName "DataGatewayPoC-2" -RecoveryKey $secureRecoveryKey -OverwriteExistingGateway -Debug -Verbose -GatewayClusterId "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

      Debug Logs:

      # Excerpt from Add-DataGatewayClusterMember debug logs
      VERBOSE: Request Uri: https://api.powerbi.com/v2.0/myorg/me/gatewayRegions
      VERBOSE: Status Code: OK (200)
      VERBOSE: Request Uri: https://wabi-west-europe-redirect.analysis.windows.net/v2.0/myorg/me/gatewayClusters?$expand=permissions,memberGateways
      VERBOSE: Status Code: OK (200)
      Add-DataGatewayClusterMember: Gateway cluster xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx was not found

      Expected Behavior:
      The Add-DataGatewayClusterMember command should respect the -RegionKey parameter and correctly find the GatewayClusterId in the specified region.

      Actual Behavior:
      The command fails with an error message indicating that the GatewayClusterId was not found.

       

      hackcrr

      If this post helps, then please consider Accept it as the solution and kudos to this post to help the other members find it more quickly

  • Anonymous's avatar
    Anonymous
    Not applicable

    Thank you hackcrr 

    Hi, mikeeq 

    I apologize for my late reply. I did a complete test based on the code you are using. During the test, the gateway module honors the region parameter. Here are the results of my tests:

    First, I updated my PowerShell to version 7.4.3 using the following commands:

     

    winget source reset --force
    winget source update
    winget install --id Microsoft.Powershell --source winget

     

    I installed the gateway module using the following command:

     

    Install-Module -Name OnPremisesDataGatewayHAMgmt
    Install-Module -Name DataGateway
    

     

    Import the gateway module in PowerShell and view the gateway version:

     

    Import-Module DataGateway
    (Get-Module DataGateway).version

     

    Next log into my gateway account:

     

    Login-DataGatewayServiceAccount

     

     

    (Get-Item "C:\Program Files\On-premises data gateway\Microsoft.PowerBI.EnterpriseGateway.exe").VersionInfo.FileVersion

     

    Use the following command to get the zone of the gateway to see which zone my current gateway is installed in:

     

    Get-DataGatewayRegion

     

    Add a gateway cluster member to the gateway region:

     

    Add-DataGatewayClusterMember -GatewayClusterId xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx -GatewayName datagatewaypoc-2 -RecoveryKey $secureRecoveryKey -OverwriteExistingGateway -RegionKey uksouth

     

    You can see that we were also able to add the gateway member without any problems when we specified the RegionKey uksouth.

    Use your debug command below to test that the Power BI API automatically redirects to the specified region when you add a gateway cluster member:

    PS C:\> Add-DataGatewayClusterMember -RegionKey uksouth -GatewayName "DataGatewayPoC-4" -RecoveryKey $secureRecoveryKey -OverwriteExistingGateway -Debug -Verbose -GatewayClusterId "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

    Use the following command to get the just added gateway cluster:

     get-dataGatewayCluster -RegionKey uksouth -Verbose -Debug

    During this entire test, the gateway module worked fine. You can try to add the gateway cluster members again using PowerShell commands. Or you can perform the operation in another computer.
    In addition to this, you can choose an alternative solution: add the gateway cluster using the local data gateway software.

     

     

    Best Regards

    Jianpeng Li

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

     

     

     

     

     

     

     

     

    • mikeeq's avatar
      mikeeq
      Frequent Visitor

      Hi Anonymous,

       

      The problem is when you are trying to add a member to the cluster which is deployed in non-default region, as far as I can see you are adding the member to the cluster which is your default one - UK South, try another region 😉

      • Anonymous's avatar
        Anonymous
        Not applicable

        Hi, mikeeq 

        Thanks for your reply. As you mentioned, when I add my gateway member to another region, it fails to add.

        Since my current main gateway is located in westus3, when I add members to this main gateway, I specify other regions, such as westeurope.
        Then when we execute the current command, Power BI will try to search for the current main gateway in the westeurope region, but in fact, our main gateway does not exist in the current region, so 404 not found appears.

         

         

         

         

        Best Regards

        Jianpeng Li

        If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

         

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi, mikeeq 

    Do you have the GatewayClusterId used by the check, and the gateway cluster is indeed deployed in that region. An incorrect GatewayClusterId is causing the current issue.

     

     

    Best Regards

    Jianpeng Li

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

    • mikeeq's avatar
      mikeeq
      Frequent Visitor

      Hi Anonymous,

       

      GatewayClusterId is for sure correct, I double-checked that multiple times, I'm pretty sure it's a bug in the PS module itself.

  • hackcrr's avatar
    hackcrr
    Memorable Member

    Hi, mikeeq 

    First, ensure that you are using the latest version of the DataGateway module. You can update the module using the following command:

     

    Update-Module DataGateway

     

    Double-check the syntax and ensure that the

     

    -RegionKey

     

     parameter is correctly specified. PowerShell parameters are case-sensitive, so make sure you're using the exact region key.

    As a temporary workaround, you can try deploying the gateway cluster in the default Power BI tenant region if possible. If this is not feasible, continue using the manual process without the RegionKey parameter until a fix is available.

    Here are examples of how you might use the commands with and without the -RegionKey parameter:

    Without -RegionKey Parameter (Default Region)

     

    $secureRecoveryKey = ConvertTo-SecureString "YourRecoveryKey" -AsPlainText -Force
    Add-DataGatewayClusterMember -GatewayClusterId "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -GatewayName "datagatewaypoc-2" -RecoveryKey $secureRecoveryKey -OverwriteExistingGateway

     

    With -RegionKey Parameter (Specific Region)

     

    $secureRecoveryKey = ConvertTo-SecureString "YourRecoveryKey" -AsPlainText -Force
    Add-DataGatewayClusterMember -GatewayClusterId "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -GatewayName "datagatewaypoc-2" -RecoveryKey $secureRecoveryKey -OverwriteExistingGateway -RegionKey "uksouth"

     

    Try using the Get-DataGatewayCluster command with the -RegionKey parameter to see if it works correctly:

     

    Get-DataGatewayCluster -RegionKey "uksouth"

     

    If the -RegionKey parameter works with other commands, it indicates that the issue is specific to the Add-DataGatewayClusterMember command.

     

     

    hackcrr

    If this post helps, then please consider Accept it as the solution and kudos to this post to help the other members find it more quickly

     

    • mikeeq's avatar
      mikeeq
      Frequent Visitor

      "ensure that you are using the latest version of the DataGateway module."

      It's mentioned in the first sentence that I'm using the latest version of the DataGateway module.

      "If the -RegionKey parameter works with other commands, it indicates that the issue is specific to the Add-DataGatewayClusterMember command."

      `-RegionKey` parameter works with other commands without any issues, there's only a problem with `Add-DataGatewayClusterMember` and maybe wth `Remove-DataGatewayClusterMember`, but as `add member` command doesn't work, I wasn't able to test the `remove member` command