Data flow/ Data Mart/ Data verse for Enterprise dataset ?

[Shadaiv]

Hi Experts. 

Need your help to understand the best practice for the Enterprise dataset creation.

We want to create a Power BI dataset which should have 80% of data required all the reporting in our organisation. This dataset would have many dimenstions and fact tables. For examples tables like Date, Employee, Cost,  Training Fact, Leave Fact

We are thinking 80% of the data would be availble in this dataset and 20% would be report specific. Also Users/ developers should be able to add calculated columns on top of this enterprise dataset.

I have been reading all the new features Data flow, Data mart and Data verse, but not sure which feature or combination of feature can be used in this scenario. 

Please advice on this. 

[otravers]

I wrote an entry to position these various artifacts, please have a look and let me know if you have more specific questions:

https://www.oliviertravers.com/power-bi-dataset-dataflow-datamart-dataverse-sql-server/

[Shadaiv]

Thank you much  @otravers for sharing this. 
I understand all the components now. 

For my requirement, I am confused between data flow and Data mart. Should i just create data flows and use it in the Power BI dataset. Or should i create a datamart which is the combination of data flow+database+dataset. 

We are using the SQL on premises data base.

[otravers · Accepted solution]

For your requirement I'd do a combination of:

1. Dataflows for ETL

2. Promoted dataset

3. Live connection children datasets

Feel free to experiment with datamarts to see if they could be part of the solution between the dataflows and promoted dataset, but be aware that they're in Preview and really not suitable for production at the moment.

[solvisig]

Well written article @otravers - Kudos for that!

I want to add to this one limitation that I have come across while trying out dataflows,datamarts and datasets for large dataset like this.
When importing huge chunks for data with incremental refreshes into powerbi dataflow and datarmart, you have no controll over the partitions automaticall created by PowerBI and therefore need to load all the data at once and cannot refresh individual partitions seperately.

For me this was the reason I ended up using dataset for this same purpose.