Dataflow Import and Merging Best Practices Question

[arock-well]

What is the best practices recommendation when creating dataflows and merging tables within that dataflow? I'm using on-prem SQL Server tables as the datasource.

1. Is it best to import data and do all the merging within one dataflow?

2. Is it best to import data into one dataflow and then create a separate dataflow (with links back to the original dataflow) where I do all the merging of those tables there?

3. Something else?

[v-henryk-mstf]

Hi @arock-well ,

It is not recommended to do all merging in one data stream. It is recommended to create a new data stream for each source (one for local and one for cloud) and then create a third data stream to merge/compute the two data sources.

Also if you need to perform merges between tables, it is recommended to use linked tables to create a data stream that allows you to refer to an existing table defined in another data stream in a read-only manner.

For more details, you can read below document:

Premium features of dataflows - Power BI | Microsoft Learn

Creating a dataflow - Power BI | Microsoft Learn

Best Regards,
Henry

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[arock-well]

@v-henryk-mstf  Thanks for that. I don't have any cloud based data, just on-prem SQL datasources.

With just one datasource as I have, do you still recommend a separate dataflow to do the merging and computing from the first dataflow that imports the data?

Also, what is the recommended maximum number of tables and merged queries that should be in one dataflow?