Dataflow, load to an existing Datamart

[Mimoune_Djouall]

Currently when you create a dataflow, it give you the option to either create a new dataflow or a new datamart, which is a great option for a new user as you get a full Online experience from Data Prep, Modelling to Final reports, and a good option to start a new standalone report

but for existing users with a bit of experience, we prefer another Workflow, give us an option to load a dataflow to an existing Datamart, we want to use a Datamart as a central Database to store a lot of tables for a specific Business Topic, Currently the only way to do it is to create a separate dataflow and load data from there, the problem is, if you have a Datamart with 100 Tables, and only one table has changed, you need to reload all the 99 tables, that's not a very good usage of PowerBI resources Neither for customers nor Microsoft.

When Loading Data from Dataflow to Datamart, we prefer those option in order of Priority.

• Overwrite
• Incremental refresh
• append
• Upsert using PK etc

This is Microsoft opportunity to redefined ETL for mass market, and blur the line between Enterprise ETL and Self service Data preparation.

[fbcideas_migusr]

Allowing dataflows to choose their destination simplifies the story for users:

• a dataflow is a way to get data from source systems into a storage destination.
• you can pick either basic datalake storage (CSVs + manifest/typing) or richer relational database storage (Azure SQL DB).

[steven_fuller1]

Mimoune, are you referring to previously existing Dataflows as well? or just new dataflows?

Personally I think what you're suggesting would be great, especially for an existing dataflow that someone wants to use in a datamart. I'm sure I'm not the only one who already has many active dataflows that I would love to be able to work with in a datamart, but don't really want to duplicate all of them just for the datamart capabilities.

[Mimoune_Djouall]

@ Steven Fuller

both the idea is to decouple data from datamart for maximum flexibility, now if you want to refresh a table, you have to refresh the whole model, that does not work in any serious scenario