Error when refreshing dataflow

paulenomial — Wed, 19 Jun 2024 10:35:17 GMT

Hi folks,

I've built a dataflow which takes a few tables from a lakehouse in another workspace and loads one of them into a warehouse after some transformation. When I look at the table in the dataflow editor the table will load fine (albeit after a 1m30s delay) but when I try to publish or refresh the dataflow I get this error:

inventtrans: Error Code: Mashup Exception Error, Error Details: Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: We're sorry, an error occurred during evaluation. Details: (Request ID: f15df4a5-85a0-43f3-9190-e00ef672f1c4).

The dataflow isn't tremendously complex, it just merges queries from a couple tables then adds a few calculated columns in to concatenate some data. Nothing has been done to break query folding from the lakehouse. Does anyone know what this could be?

I've tried deleting the query and rebuilding it and I've tried deleting the entire dataflow and rebuilding that as well to no avail.

Re: Error when refreshing dataflow

Anonymous — Thu, 20 Jun 2024 09:28:53 GMT

Hi paulenomial ,

Thanks for using Fabric Community. This might be an intermittent issue, can you please re try it again?
Please let me know incase of further queries.

Re: Error when refreshing dataflow

paulenomial — Thu, 20 Jun 2024 13:34:06 GMT

Hi Anonymous ,

I'm afraid I've tried multiple times over the last few days and it doesn't appear to be going away.

Thanks,
Paul.

Re: Error when refreshing dataflow

pqian_MSFT — Thu, 20 Jun 2024 19:02:21 GMT

I see that the query didn't fold and it attempted to buffer all the rows locally.

can you right click on the final entity and select "view query plan", what do you see there?

Re: Error when refreshing dataflow

paulenomial — Fri, 21 Jun 2024 07:25:54 GMT

Hi pqian_MSFT ,

The green indicator on the rows indicates that it is folding however the final step takes approximately 1m 30s to load but the step prior takes only about 20s despite both steps being a concatenation of two fields.

See query plan below along with the native data source query:

select [_].[costamountadjustment] as [costamountadjustment], [_].[costamountoperations] as [costamountoperations], [_].[costamountphysical] as [costamountphysical], [_].[costamountposted] as [costamountposted], [_].[dateexpected] as [dateexpected], [_].[datefinancial] as [datefinancial], [_].[datephysical] as [datephysical], [_].[inventdimid] as [inventdimid], [_].[inventtransorigin] as [inventtransorigin], [_].[invoiceid] as [invoiceid], [_].[itemid] as [itemid], [_].[packingslipid] as [packingslipid], [_].[qty] as [qty], [_].[voucher] as [voucher], [_].[voucherphysical] as [voucherphysical], [_].[dataareaid] as [dataareaid], [_].[referenceid] as [referenceid], [_].[referencecategory] as [referencecategory], [_].[configid] as [configid], [_].[inventbatchid] as [inventbatchid], [_].[inventcolorid] as [inventcolorid], [_].[inventlocationid] as [inventlocationid], [_].[inventsiteid] as [inventsiteid], [_].[inventsizeid] as [inventsizeid], [_].[inventstatusid] as [inventstatusid], [_].[inventstyleid] as [inventstyleid], [_].[wmslocationid] as [wmslocationid], convert(nvarchar(4000), [_].[_FKinventlocation]) as [_FKinventlocation], convert(nvarchar(4000), [_].[_FKinventsite]) as [_FKinventsite], convert(nvarchar(4000), [_].[_FKinventtable]) as [_FKinventtable] from ( select [_].[costamountadjustment] as [costamountadjustment], [_].[costamountoperations] as [costamountoperations], [_].[costamountphysical] as [costamountphysical], [_].[costamountposted] as [costamountposted], [_].[dateexpected] as [dateexpected], [_].[datefinancial] as [datefinancial], [_].[datephysical] as [datephysical], [_].[inventdimid] as [inventdimid], [_].[inventtransorigin] as [inventtransorigin], [_].[invoiceid] as [invoiceid], [_].[itemid] as [itemid], [_].[packingslipid] as [packingslipid], [_].[qty] as [qty], [_].[voucher] as [voucher], [_].[voucherphysical] as [voucherphysical], [_].[dataareaid] as [dataareaid], [_].[referenceid] as [referenceid], [_].[referencecategory] as [referencecategory], [_].[configid] as [configid], [_].[inventbatchid] as [inventbatchid], [_].[inventcolorid] as [inventcolorid], [_].[inventlocationid] as [inventlocationid], [_].[inventsiteid] as [inventsiteid], [_].[inventsizeid] as [inventsizeid], [_].[inventstatusid] as [inventstatusid], [_].[inventstyleid] as [inventstyleid], [_].[wmslocationid] as [wmslocationid], [_].[dataareaid] + [_].[inventlocationid] as [_FKinventlocation], [_].[dataareaid] + [_].[inventsiteid] as [_FKinventsite], [_].[dataareaid] + [_].[itemid] as [_FKinventtable] from ( select [$Outer].[costamountadjustment], [$Outer].[costamountoperations], [$Outer].[costamountphysical], [$Outer].[costamountposted], [$Outer].[dateexpected], [$Outer].[datefinancial], [$Outer].[datephysical], [$Outer].[inventdimid], [$Outer].[inventtransorigin], [$Outer].[invoiceid], [$Outer].[itemid], [$Outer].[packingslipid], [$Outer].[qty], [$Outer].[voucher], [$Outer].[voucherphysical], [$Outer].[dataareaid], [$Outer].[referenceid], [$Outer].[referencecategory], [$Inner].[configid], [$Inner].[inventbatchid], [$Inner].[inventcolorid], [$Inner].[inventlocationid], [$Inner].[inventsiteid], [$Inner].[inventsizeid], [$Inner].[inventstatusid], [$Inner].[inventstyleid], [$Inner].[wmslocationid] from ( select [$Outer].[costamountadjustment], [$Outer].[costamountoperations], [$Outer].[costamountphysical], [$Outer].[costamountposted], [$Outer].[dateexpected], [$Outer].[datefinancial], [$Outer].[datephysical], [$Outer].[inventdimid], [$Outer].[inventtransorigin], [$Outer].[invoiceid], [$Outer].[itemid], [$Outer].[packingslipid], [$Outer].[qty], [$Outer].[voucher], [$Outer].[voucherphysical], [$Outer].[dataareaid], [$Inner].[referenceid], [$Inner].[referencecategory] from ( select [costamountadjustment], [costamountoperations], [costamountphysical], [costamountposted], [dateexpected], [datefinancial], [datephysical], [inventdimid], [inventtransorigin], [invoiceid], [itemid], [packingslipid], [qty], [voucher], [voucherphysical], [dataareaid] from [dataverse_production_cds2_workspace_unq239aaf6e46184884942eb81311b62].[dbo].[inventtrans] as [$Table] ) as [$Outer] left outer join ( select [$Outer].[inventtransid] as [inventtransid], [$Outer].[referenceid] as [referenceid], [$Outer].[dataareaid] as [dataareaid2], [$Outer].[recid] as [recid], [$Inner].[LocalizedLabel] as [referencecategory] from ( select [referencecategory], [inventtransid], [referenceid], [dataareaid], [recid] from [dataverse_production_cds2_workspace_unq239aaf6e46184884942eb81311b62].[dbo].[inventtransorigin] as [$Table] ) as [$Outer] left outer join ( select [_].[Option], [_].[LocalizedLabel], [_].[ExternalValue] from [dataverse_production_cds2_workspace_unq239aaf6e46184884942eb81311b62].[dbo].[GlobalOptionsetMetadata] as [_] where ([_].[EntityName] = 'inventtransorigin' and [_].[EntityName] is not null) and ([_].[OptionSetName] = 'referencecategory' and [_].[OptionSetName] is not null) ) as [$Inner] on ([$Outer].[referencecategory] = [$Inner].[Option] or [$Outer].[referencecategory] is null and [$Inner].[Option] is null) ) as [$Inner] on ([$Outer].[inventtransorigin] = [$Inner].[recid] or [$Outer].[inventtransorigin] is null and [$Inner].[recid] is null) ) as [$Outer] left outer join ( select [configid] as [configid], [inventbatchid] as [inventbatchid], [inventcolorid] as [inventcolorid], [inventdimid] as [inventdimid2], [inventlocationid] as [inventlocationid], [inventsiteid] as [inventsiteid], [inventsizeid] as [inventsizeid], [inventstatusid] as [inventstatusid], [inventstyleid] as [inventstyleid], [wmslocationid] as [wmslocationid], [dataareaid] as [dataareaid2] from [dataverse_production_cds2_workspace_unq239aaf6e46184884942eb81311b62].[dbo].[inventdim] as [$Table] ) as [$Inner] on (([$Outer].[inventdimid] = [$Inner].[inventdimid2] or [$Outer].[inventdimid] is null and [$Inner].[inventdimid2] is null) and ([$Outer].[dataareaid] = [$Inner].[dataareaid2] or [$Outer].[dataareaid] is null and [$Inner].[dataareaid2] is null)) ) as [_] ) as [_]

Re: Error when refreshing dataflow

pqian_MSFT — Fri, 21 Jun 2024 17:21:52 GMT

This query plan should not be buffering at all.

What is the entity name? The one causing issue is called "inventtrans".

Re: Error when refreshing dataflow

paulenomial — Mon, 24 Jun 2024 07:29:44 GMT

Yes the entity is called inventtrans.

Re: Error when refreshing dataflow

Anonymous — Mon, 01 Jul 2024 09:12:51 GMT

Hi paulenomial ,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
In case if you have any resolution please do share that same with the community as it can be helpful to others .

In case if you are still facing the issue, please reach out to our support team so they can do a more thorough investigation on why this it is happening: Link

After creating a Support ticket please provide the ticket number as it would help us to track for more information.

Hope this helps. Please let us know if you have any other queries.

Re: Error when refreshing dataflow

Anonymous — Wed, 03 Jul 2024 06:43:13 GMT

Hi paulenomial ,

We haven’t heard from you on the last response and was just checking back to see if you got a chance to open a support ticket.
After creating a Support ticket please provide the ticket number as it would help us to track for more information.

Thanks

topic Re: Error when refreshing dataflow in Dataflow

Error when refreshing dataflow

Re: Error when refreshing dataflow

Re: Error when refreshing dataflow

Re: Error when refreshing dataflow

Re: Error when refreshing dataflow

Re: Error when refreshing dataflow

Re: Error when refreshing dataflow

Re: Error when refreshing dataflow

Re: Error when refreshing dataflow