Automatically Detect New MongoDB Collections in Power BI

[raaan12]

Hi everyone,

I’m working with Power BI and using the MongoDB connector to import my data. 

My question is: if a new collection is later added to the MongoDB database, Power BI doesn’t seem to detect it automatically. I have to go back into Power BI and manually load this new collection myself.

Is there a way to configure Power BI to automatically detect and load any new collections that get added to the MongoDB database over time?

Thanks in advance!

[rohit1991 · Accepted solution]

Hi @raaan12 ,

In Power BI, when using the MongoDB connector, new collections are not automatically detected or loaded after the initial connection setup. This is because Power BI imports data based on a predefined query or selection made during the initial data load. Power BI does not dynamically scan the MongoDB database schema for changes, such as newly added collections, in subsequent refreshes.

If you want Power BI to include new collections as they are added, you would need to implement a more dynamic data retrieval approach. One workaround is to use Power Query's MongoDB driver with a custom script that queries the database for all collections (using system metadata like db.getCollectionNames() if you're working outside Power BI in an ETL tool or script). However, this isn't directly supported inside Power BI without external scripting (e.g., via Python or a dataflow API integration).

In short, Power BI does not currently support automatic detection and import of new MongoDB collections, and you will need to manually update the data model or use a more flexible ETL process outside of Power BI for dynamic schema changes. Let me know if you’d like help designing an automated process outside Power BI to handle this.

[danextian]

Hi @raaan12 

You can run a python script to detect new collections. While this is possible, dynamically autoloading the new collections to the model isn't.

[Vijay_Chethan]

Hello raaan12,
Unfortunately, Power BI doesn't automatically detect new collections added to a MongoDB database. but could achieve it using power automate and python script and power bi rest api

if it helps , please mark as solution

[raaan12]

Hi Vijay_Chethan,

Thanks so much for your suggestion. That's an interesting approach!

I'm trying to understand the Power Automate part of your suggestion better. Based on what I'm gathering, would the flow look something like this:

1. Trigger: Set up a Power Automate flow to run on a schedule (e.g., daily).

2. Action 1: Run Python Script: The flow executes a Python script that connects to MongoDB, identifies new collections, and perhaps outputs their names.

3. Action 2: Get New Collection List: Power Automate receives this list of new collection names from the Python script.

4. Action 3: Interact with Power BI REST API: For each new collection, Power Automate makes calls to the Power BI REST API. This is the part I'm most curious about – would this involve programmatically updating an existing dataset's M query, or perhaps triggering a dataflow that's designed to pick up new sources, or something else to get Power BI to recognize and load data from these new collections?

Otherwise, do you know any guided tutorials, blog posts, or examples that demonstrate a similar setup, particularly the interaction between Power Automate and the Power BI REST API for discovering and incorporating new data sources like MongoDB collections in this manner?

Any pointers or resources you could share on how Action 3 might be practically achieved would be immensely helpful.

Thanks again!

[v-sshirivolu]

Hi @raaan12 ,
Thank you for the helpful response @danextian !

To help you connect MongoDB with Power BI and automate your workflows, we’ve pulled together some useful resources. The Power BI REST API docs show how to manage datasets, refreshes, and reports programmatically. If you're using MongoDB, the official connector guide explains how to connect it to Power BI. For automation, this blog post walks through using Power Automate to trigger API actions. Finally, Hevo’s guides break down different connection methods and real-world API use cases.

If this post helps, please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.


Regards,
Sreeteja.

[v-sshirivolu]

Hi @raaan12 

I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If our response has addressed your query, please Accept it as a solution so that other community members can find it easily.

Thank you.