Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
HamidBee
Impactful Individual
Impactful Individual

Utilizing Dataflows for Recurring Excel File Updates in OneDrive and Upsert Operations in Dataverse

I am currently investigating the use of Power Automate and Dataflows for handling a scenario in which Excel files are regularly uploaded to a OneDrive folder, with each file containing a table that needs to be upserted to a Dataverse table. After a revealing exchange in a previous discussion, I have compiled a list of queries regarding the deployment and capability of Dataflows for this context:

 

  1. Is it possible to configure Dataflows to automatically sync with a specified OneDrive folder on a continuous basis, so that any edit, deletion, or addition to the rows in any Excel file within this folder will trigger the Dataflow?

  2. What are the limitations, if any, on the frequency of Dataflows executions within a day? Are Dataflows primarily triggered on a set schedule, or can they be configured to activate in response to changes detected in the OneDrive folder?

  3. In comparison to the upsert function provided by the Dataverse Web API, how does the performance of Dataflows measure up, especially when dealing with large datasets containing millions of rows?

  4. Considering the advice previously given, which involved identifying the table name in the 'List rows present in a table' action, how can one manage tables with names that are not consistent or prone to change? Is there a flexible approach to referencing these tables without resorting to hardcoded names?

The overarching objective is to establish a dependable and expandable system that ensures efficient data transfer from Excel to Dataverse, capable of catering to datasets ranging from several hundreds to millions of rows.

I eagerly await your suggestions and expert insights. Thank you for your contribution to this discussion.

1 REPLY 1
lbendlin
Super User
Super User

1. yes

2. none if you use the API

3. As long as you keep the dataflow nimble (no fancy transforms or attempts at merges) performance  should "measure up"  - whatever that means in your environment. It sure outperforms Excel.

4. I'm pretty sure you can specify dynamic content in that dialog.  It will become hairy, however, when these tables have different structure.

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

AugPowerBI_Carousel

Power BI Monthly Update - August 2024

Check out the August 2024 Power BI update to learn about new features.

August Carousel

Fabric Community Update - August 2024

Find out what's new and trending in the Fabric Community.