Forum Discussion

amaaiia's avatar
amaaiia
Skilled Sharer
2 months ago
Solved

Best way to use VS Code + GitHub Copilot with Microsoft Fabric?

Until now, I have been working with Microsoft Fabric mainly through its visual interface. I now want to start using VS Code for development in Fabric, leveraging GitHub Copilot to improve productivity.

Currently, I have an Azure DevOps repository synchronized with several Microsoft Fabric workspaces (data workspace, notebooks/pipelines workspace, and reports workspace), and I’m working with this repo in VS Code. I’ve also seen that there is a Microsoft Fabric extension that allows connecting to workspaces in VS Code.

 

My goal is to understand what the best practice is for working with VS Code in this context, specifically for developing and managing:

  • Notebooks
  • Pipelines
  • Environments
  • Other Fabric artifacts

And especially how to effectively leverage GitHub Copilot for this workflow, for example:

  • Improving existing notebooks
  • Creating new notebooks based on existing ones (or from scratch)
  • Creating and modifying pipelines

What is the recommended way to structure this development workflow using VS Code, Fabric and Azure Repos?

  • Hi amaaiia,

    That’s a very relevant question, especially as more teams are trying to bring Fabric development closer to a proper software engineering workflow.

    The way I usually think about it is :

    • Use Fabric Git integration as the main bridge between the Fabric workspace and Azure Repos.
    • Use VS Code as the local development environment once the Fabric items are synced to the repo.
    • Use GitHub Copilot mainly to improve and accelerate code-heavy artifacts, especially notebooks, PySpark, SQL scripts, and pipeline JSON/TMDL where applicable.
    • Keep the actual validation and execution inside Fabric, because some artifacts still depend heavily on the Fabric runtime, workspace context, connections, lakehouse bindings, and permissions.

    For notebooks, the workflow is usually quite smooth :
    Fabric workspace > Git sync > Azure Repos > VS Code > Copilot-assisted changes > Pull request > sync/deploy back to Fabric.

    For pipelines and other Fabric artifacts, I would be more careful. Copilot can help review or modify the underlying JSON, but I would avoid treating all Fabric items like fully portable code yet. Some bindings, connections, parameters, and environment-specific references may still need to be validated directly in Fabric.

    So my recommended pattern would be :

    1. Use Azure Repos as the source-controlled backbone.
    2. Structure the repo by workspace/domain/artifact type.
    3. Use branches and PRs for changes.
    4. Let Copilot assist with notebook creation, refactoring, documentation, SQL/PySpark improvements, and JSON review.
    5. Validate everything in a dedicated Fabric dev workspace before promotion.
    6. Use deployment pipelines or scripted deployment only after checking bindings and environment-specific settings.

    In short : VS Code + Copilot is very useful for the development layer, but Fabric should still remain the validation and execution layer, especially for pipelines and workspace-bound artifacts.

    Hope this helps frame the workflow a bit.

    Some references :

     

  •  I do it pretty much like you do. The way I do it is as follows

     

    Github/VSCode:

    - I use github copilot (and a repo) to create SQL code for (mainly) ETL and Python notebooks. And to make sure i have version control while developping .. aka the real coding stuff (Github co-pilot can now also do DAX if i recall correctly)

    - vscode (connected to github) i have connected to my fabric instance so i can directly run sql code agains the lakehouse sql enpoint and the warehouse

    - I have created an MD file that I use in a repo, that states what our naming conventions are and any default joins, which i know will come back with any dataset I built.

    - Github copilot, with vscode intellisense, is extremely powerful in predicting my next few lines, especially in SQL , but you need to be cautious, for instance we always use decimal (18,6) but a few times already co-pilot decided to hallucinate it into a Decimal (18,10)

     

    Microsoft:

    - I use MS Co-pilot for general fabric questions, for instance if I am using an activity which is not top of mind for me, but also when i get an error when running a pipeline, and i don't instantly see what the issue is.

    - I Use the fabric User Interface for creating actual pipelines, with thier activities and fysical notebooks

     

    I cannot remember the last time I actually editted an underlying pipeline json.

9 Replies

  • Hi amaaiia,

    That’s a very relevant question, especially as more teams are trying to bring Fabric development closer to a proper software engineering workflow.

    The way I usually think about it is :

    • Use Fabric Git integration as the main bridge between the Fabric workspace and Azure Repos.
    • Use VS Code as the local development environment once the Fabric items are synced to the repo.
    • Use GitHub Copilot mainly to improve and accelerate code-heavy artifacts, especially notebooks, PySpark, SQL scripts, and pipeline JSON/TMDL where applicable.
    • Keep the actual validation and execution inside Fabric, because some artifacts still depend heavily on the Fabric runtime, workspace context, connections, lakehouse bindings, and permissions.

    For notebooks, the workflow is usually quite smooth :
    Fabric workspace > Git sync > Azure Repos > VS Code > Copilot-assisted changes > Pull request > sync/deploy back to Fabric.

    For pipelines and other Fabric artifacts, I would be more careful. Copilot can help review or modify the underlying JSON, but I would avoid treating all Fabric items like fully portable code yet. Some bindings, connections, parameters, and environment-specific references may still need to be validated directly in Fabric.

    So my recommended pattern would be :

    1. Use Azure Repos as the source-controlled backbone.
    2. Structure the repo by workspace/domain/artifact type.
    3. Use branches and PRs for changes.
    4. Let Copilot assist with notebook creation, refactoring, documentation, SQL/PySpark improvements, and JSON review.
    5. Validate everything in a dedicated Fabric dev workspace before promotion.
    6. Use deployment pipelines or scripted deployment only after checking bindings and environment-specific settings.

    In short : VS Code + Copilot is very useful for the development layer, but Fabric should still remain the validation and execution layer, especially for pipelines and workspace-bound artifacts.

    Hope this helps frame the workflow a bit.

    Some references :

     

    • RajeshM's avatar
      RajeshM
      Advocate II

      Thanks for the explanation. I can relate the same to github.com as well.

       

  • Use OneDrive instead of GitHub to store, catalogue and share code and projects with others in a password protected way.

  • v-ssriganesh's avatar
    v-ssriganesh
    Community Support

    Hello amaaiia,

    We hope you're doing well. Could you please confirm whether your issue has been resolved or if you're still facing challenges? Your update will be valuable to the community and may assist others with similar concerns.

    Thank you.

  • v-ssriganesh's avatar
    v-ssriganesh
    Community Support

    Hello amaaiia,

    Hope everything’s going great with you. Just checking, the issue been resolved or are you still running into problems? Sharing an update can really help others facing the same thing.

    Thank you.

     

  • smeetsh's avatar
    smeetsh
    Continued Contributor

     I do it pretty much like you do. The way I do it is as follows

     

    Github/VSCode:

    - I use github copilot (and a repo) to create SQL code for (mainly) ETL and Python notebooks. And to make sure i have version control while developping .. aka the real coding stuff (Github co-pilot can now also do DAX if i recall correctly)

    - vscode (connected to github) i have connected to my fabric instance so i can directly run sql code agains the lakehouse sql enpoint and the warehouse

    - I have created an MD file that I use in a repo, that states what our naming conventions are and any default joins, which i know will come back with any dataset I built.

    - Github copilot, with vscode intellisense, is extremely powerful in predicting my next few lines, especially in SQL , but you need to be cautious, for instance we always use decimal (18,6) but a few times already co-pilot decided to hallucinate it into a Decimal (18,10)

     

    Microsoft:

    - I use MS Co-pilot for general fabric questions, for instance if I am using an activity which is not top of mind for me, but also when i get an error when running a pipeline, and i don't instantly see what the issue is.

    - I Use the fabric User Interface for creating actual pipelines, with thier activities and fysical notebooks

     

    I cannot remember the last time I actually editted an underlying pipeline json.

  • v-csrikanth's avatar
    v-csrikanth
    Community Support

    Hi  amaaiia 
    We would like to inquire whether have you got the chance to check the solutions provided by other users in commiunity to resolve the issue. We hope the information provided helps to clear the query. Should you have any further queries, kindly feel free to contact the Microsoft Fabric community.


    Thanks, 
    Srikanth Cheri
    Community Support Team