Advice on custom connector source code organization

[BenLambert]

I'm building multiple Power Query connectors in VS Code (on Windows) using the Power Query SDK. I'd like to keep all the connectors in a single repo (in a folder-per-connector), and I'd also like to share some of the boilerplate/generic code across the connectors without having multiple copies of that code.  I know I can create separate folders for each connector project, but don't see an easy way to share code between the connectors or an easy way to build each connector without opening each folder separately and hitting Ctrl+Shift+B.

Is this a smart approach, or am I overcomplicating my life and should just live with a separate folders per project and copies of the various helper .pqm modules?

I was envisioning a folder structure like this:

my-powerbi-connectors/
│
├── .git/
├── .gitignore
├── README.md
│
├── Shared/
│ ├── Table.ChangeType.pqm
│ └── Table.GenerateByPage.pqm
│ └── OAuth.StartLogin.pqm

│ └── other shared code in .pqm files
│ ...
│
├── Connector1/
│ ├── Connector1.pq
│ └── Connector1.query.pq
│ ...
│
├── Connector2/
│ ├── Connector2.pq
│ └── Connector2.query.pq

[v-kpoloju-msft · Accepted solution]

Hi @BenLambert,

Thank you for reaching out to the Microsoft fabric community forum. I have identified several alternative workarounds that may assist in resolving the thread.

As of now, Microsoft does not officially support importing shared .pqm modules from outside a single connector folder in the Power Query SDK.

You're on the right path by aiming to organize multiple Power Query custom connectors in a single repository with shared code. This approach is beneficial, especially when managing multiple connectors with similar functionalities.

Currently, the Power Query SDK for Visual Studio Code does not officially support referencing shared .pqm files from outside a single connector's project folder. Each connector is intended to be self-contained, as noted in the SDK documentation:

However, many developers have worked around this limitation by:

• Using symbolic links to include a shared Shared/ folder inside each connector directory.
• Writing simple build scripts that copy shared .pqm modules into each connector folder before building.

For building all connectors without manually opening each one in VS Code, consider using a PowerShell or shell script to iterate through each connector folder and invoke dotnet build. This can significantly streamline your workflow.

Additionally, you can configure a .code-workspace file to open all connector folders in one VS Code session, making development smoother.

Kinly refers to the below following documentation links for better understanding:
Power Query SDK Overview - Power Query | Microsoft Learn

If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.

Thank you for using the Microsoft Community Forum.

[Poojara_D12 · Accepted solution]

Hi @BenLambert 

Your approach of organizing multiple Power Query connectors in a single repository with a folder per connector, plus a shared folder for common boilerplate code, is conceptually a good way to keep your codebase clean and DRY (Don’t Repeat Yourself). However, Power Query’s SDK and build system currently don’t provide native support for sharing .pqm or .pq code modules across multiple projects easily, especially in terms of build automation and IntelliSense integration in VS Code. This means you typically cannot just reference shared code files in a separate folder and expect them to be automatically included or built when compiling each connector. As a result, your ideal structure—with a shared folder for common helper modules—makes sense from a source control and organization perspective but will require some manual or scripted steps. For example, you might have to copy or symlink the shared .pqm files into each connector project folder before building, or you could build a custom build script that automates this step, so you don’t have to open each project separately or manually trigger builds. Alternatively, living with separate project folders and copying helper modules is simpler but less maintainable. Overall, your planned structure is a solid foundation, but you’ll need to implement a build or sync process to share code efficiently without duplicating files in each connector folder.

[v-kpoloju-msft · Accepted solution]

Hi @BenLambert,

Thank you for reaching out to the Microsoft fabric community forum. I have identified several alternative workarounds that may assist in resolving the thread.

As of now, Microsoft does not officially support importing shared .pqm modules from outside a single connector folder in the Power Query SDK.

You're on the right path by aiming to organize multiple Power Query custom connectors in a single repository with shared code. This approach is beneficial, especially when managing multiple connectors with similar functionalities.

Currently, the Power Query SDK for Visual Studio Code does not officially support referencing shared .pqm files from outside a single connector's project folder. Each connector is intended to be self-contained, as noted in the SDK documentation:

However, many developers have worked around this limitation by:

• Using symbolic links to include a shared Shared/ folder inside each connector directory.
• Writing simple build scripts that copy shared .pqm modules into each connector folder before building.

For building all connectors without manually opening each one in VS Code, consider using a PowerShell or shell script to iterate through each connector folder and invoke dotnet build. This can significantly streamline your workflow.

Additionally, you can configure a .code-workspace file to open all connector folders in one VS Code session, making development smoother.

Kinly refers to the below following documentation links for better understanding:
Power Query SDK Overview - Power Query | Microsoft Learn

If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.

Thank you for using the Microsoft Community Forum.

[v-kpoloju-msft]

Hi @BenLambert,

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

[v-kpoloju-msft]

Hi @BenLambert,

I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.
Thank you.

[v-kpoloju-msft]

Hi @BenLambert,

I hope this information is helpful. Please let me know if you have any further questions or if you'd like to discuss this further. If this answers your question, please Accept it as a solution and give it a 'Kudos' so others can find it easily.
Thank you.