Forum Discussion
Error when refreshing dataflow
Hi folks,
I've built a dataflow which takes a few tables from a lakehouse in another workspace and loads one of them into a warehouse after some transformation. When I look at the table in the dataflow editor the table will load fine (albeit after a 1m30s delay) but when I try to publish or refresh the dataflow I get this error:
inventtrans: Error Code: Mashup Exception Error, Error Details: Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: We're sorry, an error occurred during evaluation. Details: (Request ID: f15df4a5-85a0-43f3-9190-e00ef672f1c4).
The dataflow isn't tremendously complex, it just merges queries from a couple tables then adds a few calculated columns in to concatenate some data. Nothing has been done to break query folding from the lakehouse. Does anyone know what this could be?
I've tried deleting the query and rebuilding it and I've tried deleting the entire dataflow and rebuilding that as well to no avail.
8 Replies
- AnonymousNot applicable
Hi paulenomial ,
Thanks for using Fabric Community. This might be an intermittent issue, can you please re try it again?
Please let me know incase of further queries.- paulenomialFrequent Visitor
Hi Anonymous ,
I'm afraid I've tried multiple times over the last few days and it doesn't appear to be going away.
Thanks,
Paul.
- pqian_MSFT
Microsoft Employee
I see that the query didn't fold and it attempted to buffer all the rows locally.
can you right click on the final entity and select "view query plan", what do you see there?
- paulenomialFrequent Visitor
Hi pqian_MSFT ,
The green indicator on the rows indicates that it is folding however the final step takes approximately 1m 30s to load but the step prior takes only about 20s despite both steps being a concatenation of two fields.
See query plan below along with the native data source query:
select [_].[costamountadjustment] as [costamountadjustment], [_].[costamountoperations] as [costamountoperations], [_].[costamountphysical] as [costamountphysical], [_].[costamountposted] as [costamountposted], [_].[dateexpected] as [dateexpected], [_].[datefinancial] as [datefinancial], [_].[datephysical] as [datephysical], [_].[inventdimid] as [inventdimid], [_].[inventtransorigin] as [inventtransorigin], [_].[invoiceid] as [invoiceid], [_].[itemid] as [itemid], [_].[packingslipid] as [packingslipid], [_].[qty] as [qty], [_].[voucher] as [voucher], [_].[voucherphysical] as [voucherphysical], [_].[dataareaid] as [dataareaid], [_].[referenceid] as [referenceid], [_].[referencecategory] as [referencecategory], [_].[configid] as [configid], [_].[inventbatchid] as [inventbatchid], [_].[inventcolorid] as [inventcolorid], [_].[inventlocationid] as [inventlocationid], [_].[inventsiteid] as [inventsiteid], [_].[inventsizeid] as [inventsizeid], [_].[inventstatusid] as [inventstatusid], [_].[inventstyleid] as [inventstyleid], [_].[wmslocationid] as [wmslocationid], convert(nvarchar(4000), [_].[_FKinventlocation]) as [_FKinventlocation], convert(nvarchar(4000), [_].[_FKinventsite]) as [_FKinventsite], convert(nvarchar(4000), [_].[_FKinventtable]) as [_FKinventtable] from ( select [_].[costamountadjustment] as [costamountadjustment], [_].[costamountoperations] as [costamountoperations], [_].[costamountphysical] as [costamountphysical], [_].[costamountposted] as [costamountposted], [_].[dateexpected] as [dateexpected], [_].[datefinancial] as [datefinancial], [_].[datephysical] as [datephysical], [_].[inventdimid] as [inventdimid], [_].[inventtransorigin] as [inventtransorigin], [_].[invoiceid] as [invoiceid], [_].[itemid] as [itemid], [_].[packingslipid] as [packingslipid], [_].[qty] as [qty], [_].[voucher] as [voucher], [_].[voucherphysical] as [voucherphysical], [_].[dataareaid] as [dataareaid], [_].[referenceid] as [referenceid], [_].[referencecategory] as [referencecategory], [_].[configid] as [configid], [_].[inventbatchid] as [inventbatchid], [_].[inventcolorid] as [inventcolorid], [_].[inventlocationid] as [inventlocationid], [_].[inventsiteid] as [inventsiteid], [_].[inventsizeid] as [inventsizeid], [_].[inventstatusid] as [inventstatusid], [_].[inventstyleid] as [inventstyleid], [_].[wmslocationid] as [wmslocationid], [_].[dataareaid] + [_].[inventlocationid] as [_FKinventlocation], [_].[dataareaid] + [_].[inventsiteid] as [_FKinventsite], [_].[dataareaid] + [_].[itemid] as [_FKinventtable] from ( select [$Outer].[costamountadjustment], [$Outer].[costamountoperations], [$Outer].[costamountphysical], [$Outer].[costamountposted], [$Outer].[dateexpected], [$Outer].[datefinancial], [$Outer].[datephysical], [$Outer].[inventdimid], [$Outer].[inventtransorigin], [$Outer].[invoiceid], [$Outer].[itemid], [$Outer].[packingslipid], [$Outer].[qty], [$Outer].[voucher], [$Outer].[voucherphysical], [$Outer].[dataareaid], [$Outer].[referenceid], [$Outer].[referencecategory], [$Inner].[configid], [$Inner].[inventbatchid], [$Inner].[inventcolorid], [$Inner].[inventlocationid], [$Inner].[inventsiteid], [$Inner].[inventsizeid], [$Inner].[inventstatusid], [$Inner].[inventstyleid], [$Inner].[wmslocationid] from ( select [$Outer].[costamountadjustment], [$Outer].[costamountoperations], [$Outer].[costamountphysical], [$Outer].[costamountposted], [$Outer].[dateexpected], [$Outer].[datefinancial], [$Outer].[datephysical], [$Outer].[inventdimid], [$Outer].[inventtransorigin], [$Outer].[invoiceid], [$Outer].[itemid], [$Outer].[packingslipid], [$Outer].[qty], [$Outer].[voucher], [$Outer].[voucherphysical], [$Outer].[dataareaid], [$Inner].[referenceid], [$Inner].[referencecategory] from ( select [costamountadjustment], [costamountoperations], [costamountphysical], [costamountposted], [dateexpected], [datefinancial], [datephysical], [inventdimid], [inventtransorigin], [invoiceid], [itemid], [packingslipid], [qty], [voucher], [voucherphysical], [dataareaid] from [dataverse_production_cds2_workspace_unq239aaf6e46184884942eb81311b62].[dbo].[inventtrans] as [$Table] ) as [$Outer] left outer join ( select [$Outer].[inventtransid] as [inventtransid], [$Outer].[referenceid] as [referenceid], [$Outer].[dataareaid] as [dataareaid2], [$Outer].[recid] as [recid], [$Inner].[LocalizedLabel] as [referencecategory] from ( select [referencecategory], [inventtransid], [referenceid], [dataareaid], [recid] from [dataverse_production_cds2_workspace_unq239aaf6e46184884942eb81311b62].[dbo].[inventtransorigin] as [$Table] ) as [$Outer] left outer join ( select [_].[Option], [_].[LocalizedLabel], [_].[ExternalValue] from [dataverse_production_cds2_workspace_unq239aaf6e46184884942eb81311b62].[dbo].[GlobalOptionsetMetadata] as [_] where ([_].[EntityName] = 'inventtransorigin' and [_].[EntityName] is not null) and ([_].[OptionSetName] = 'referencecategory' and [_].[OptionSetName] is not null) ) as [$Inner] on ([$Outer].[referencecategory] = [$Inner].[Option] or [$Outer].[referencecategory] is null and [$Inner].[Option] is null) ) as [$Inner] on ([$Outer].[inventtransorigin] = [$Inner].[recid] or [$Outer].[inventtransorigin] is null and [$Inner].[recid] is null) ) as [$Outer] left outer join ( select [configid] as [configid], [inventbatchid] as [inventbatchid], [inventcolorid] as [inventcolorid], [inventdimid] as [inventdimid2], [inventlocationid] as [inventlocationid], [inventsiteid] as [inventsiteid], [inventsizeid] as [inventsizeid], [inventstatusid] as [inventstatusid], [inventstyleid] as [inventstyleid], [wmslocationid] as [wmslocationid], [dataareaid] as [dataareaid2] from [dataverse_production_cds2_workspace_unq239aaf6e46184884942eb81311b62].[dbo].[inventdim] as [$Table] ) as [$Inner] on (([$Outer].[inventdimid] = [$Inner].[inventdimid2] or [$Outer].[inventdimid] is null and [$Inner].[inventdimid2] is null) and ([$Outer].[dataareaid] = [$Inner].[dataareaid2] or [$Outer].[dataareaid] is null and [$Inner].[dataareaid2] is null)) ) as [_] ) as [_]- pqian_MSFT
Microsoft Employee
This query plan should not be buffering at all.
What is the entity name? The one causing issue is called "inventtrans".