Forum Discussion

bhavyamalik1's avatar
bhavyamalik1
New Member
8 months ago
Solved

dataflow source change from aws redshift to databricks

Hi, I've some reports which use dataflows. Right now, those dataflows are using amazon redshift as the source. I want to use databricks instead. How can i change that within the dataflow? Please guid...
Need Help
Show and Tell
  • Mauro89's avatar
    Mauro89
    8 months ago

    Hi bhavyamalik1,

     

    You can change the data source in your dataflow, but it requires recreating the connection. Here's how you can try to approach it:

    Option 1: Edit directly in Power BI Service (Recommended)

    1. Go to your workspace and open the dataflow for editing
    2. In Power Query Editor, select the query connected to Redshift
    3. Click Data source settings or Configure connection
    4. Unfortunately, you cannot switch connector types (Redshift → Databricks) directly
    5. You'll need to delete the current source step and add a new Databricks connection
    6. Right-click the Source step → Delete
    7. Get Data → Databricks → enter your connection details
    8. Reapply your transformation steps (they should still be there)

    Option 2: Use the JSON file (Advanced)

    1. Open the downloaded JSON file
    2. Find the data source section (look for "Amazon Redshift" references)
    3. Replace with Databricks connection string format:
      • Change the connector from AmazonRedshift to Databricks
      • Update server, database, and authentication details
    4. This is risky - one syntax error breaks the dataflow

    Option 3: Recreate the dataflow (Safest)

    1. Create a new dataflow
    2. Connect to Databricks
    3. Copy/paste the M code from your existing queries (from the JSON or Advanced Editor)
    4. Update reports to point to the new dataflow

     

    Best regards!

    PS: If you find this post helpful consider leaving kudos or mark it as solution

Nabha-Ahmed's avatar
Nabha-Ahmed
Super User
8 months ago

Hi bhavyamalik1 

I hope you are doing well!!

You don’t need to modify the exported JSON file. Dataflow definitions aren’t designed to be edited directly, and manual changes may break the dataflow.

The correct way is to update the connection through Power Query Online:

1. Open the Power BI Service


2. Navigate to the workspace that contains the dataflow


3. Select Edit dataflow


4. In Power Query Online, open each query that currently uses Amazon Redshift


5. Update the Source step to use the Azure Databricks connector instead


6. Provide the Databricks connection details (server, HTTP path, authentication)


7. Save the dataflow and validate the refresh

 

If the schema between Redshift and Databricks is different, it’s often cleaner to:

Create a new query using Databricks as the source

Reapply the existing transformation steps

Replace the old query once validated


Editing the JSON file is not supported and is not recommended.

This approach keeps the dataflow stable, supported, and easy to maintain.

If this help you mark as "Solution" and put kudo to help other