Forum Discussion

sumanthkakarla's avatar
sumanthkakarla
Helper II
1 year ago
Solved

Parameterize a data connection using copy data activity when extracting data from GBQ to Lakehouse

Hello, I am trying to extract the data from google BQ to lakeshoue, however i have parametarized the google BD SQL, in the same way i would like to parametarize the connection name as well, how can ...
Need Help
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi sumanthkakarla,

     

    Thank you for reaching out to the Microsoft Fabric Forum Community.  Please try with below code and refine as per your requirements.

     

    $filePath = "$(Build.SourcesDirectory)/Pipeline/pipeline1.json"

    # Define the original and replacement connection strings
    $original = "Test-w3prc4skt6eu3pib5o7d4pbqqm.datawarehouse.fabric.microsoft.com"
    $replacement = "test.datawarehouse.fabric.microsoft.com"

    # Check if file exists
    if (Test-Path $filePath) {
    $content = Get-Content $filePath -Raw

    if ($content -like "*$original*") {
    Write-Host "Found target string in $filePath. Replacing..."
    $updated = $content -replace [Regex]::Escape($original), $replacement
    Set-Content -Path $filePath -Value $updated
    Write-Host "Replacement complete in $filePath"
    }
    else {
    Write-Host "Target string not found. No changes made."
    }
    }
    else {
    Write-Error "File not found: $filePath"
    }

    Please make sure to update the artifactId, workspaceId, and endpoint with the correct values according to your requirements if you are using the warehouse. For lakehouse deployments using the deployment pipelines in fabrics, the system will automatically point to the lakehouse in the target workspace if it has the same name.

     

     

    Best regards,
    Prasanna Kumar

sumanthkakarla's avatar
sumanthkakarla
Helper II
1 year ago

Thank you GilbertQ , the parametrised connection is working for me as expected, we can close the thread.

The only problem i am facing right now is creating a deployment rules for data pipeline, what i heard from Microsoft was deployment rules are not supported for data pipelines, its a manual effort once we migrate the data pipeline to stage or prod workspace to update all the pipeline parameter values.

Thank you Anonymous , like you said may be i can give it a try using  power shell script to update the pipeline parameter values.