Microsoft Fabric - Dataflow Gen2 - Merge Queries

Hi jlakshminaraya 

Thanks for using Microsoft Fabric Community.
At this time, we are reaching out to the internal team to get some help on this .
We will update you once we hear back from them.

Appreciate your patience.

Thanks

Hi Lakshmi,

It really depends. With the information provided I'm not sure that we can share that many guidelines. The underlying issue could be related to any of the following:

• Your data source is unable to fold the inner join and its subsequent expand operation
• The connector that you're using might not have the right capabilities to fold the full folding that you're trying to perform
• If you're using a gateway, perhaps try to upgrade to the latest version of the gateway
• There might be operations in your query which might be preventing the folding from happening

Below is a list of articles that might be of help in regards to understand how things can get evaluated for your query and how to try and optimize them:

• Understanding query evaluation and query folding in Power Query - Power Query | Microsoft Learn
• Query folding examples in Power Query - Power Query | Microsoft Learn
• Query folding indicators in Power Query - Power Query | Microsoft Learn
• Query plan - Power Query | Microsoft Learn
• Best practices when working with Power Query - Power Query | Microsoft Learn

If you can share some steps on how we could repro the scenario on our end, that would help us pinpoint exactly what could be happening. If perhaps you'd rather use your own dataflow, I'd recommend creating a support ticket so an engineer can take a closer look at this but, again, with the information provided it is not possible to reach a conclusion as to what might be the root cause of the issue.

jlakshminaraya if you found your merges aren't folding, the best way to deal with this is to change to the ELT pattern - create a dataflow with two STAGED entities representing the raw data to be merged, refresh it once. Then create a new dataflow and use the Dataflows connector to bring those two staged entities in - at this point they have compute from LH\SQL and you should be able to easily merge them with inner joins.

Hi jlakshminaraya 

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.
In case if you have any resolution please do share that same with the community as it can be helpful to others.
Otherwise, will respond back with the more details and we will try to help.

Thanks.