Forum Discussion

paulenomial's avatar
paulenomial
Frequent Visitor
2 years ago

Error when refreshing dataflow

Hi folks,

 

I've built a dataflow which takes a few tables from a lakehouse in another workspace and loads one of them into a warehouse after some transformation. When I look at the table in the dataflow editor the table will load fine (albeit after a 1m30s delay) but when I try to publish or refresh the dataflow I get this error:

 

inventtrans: Error Code: Mashup Exception Error, Error Details: Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: We're sorry, an error occurred during evaluation. Details: (Request ID: f15df4a5-85a0-43f3-9190-e00ef672f1c4).

 

The dataflow isn't tremendously complex, it just merges queries from a couple tables then adds a few calculated columns in to concatenate some data. Nothing has been done to break query folding from the lakehouse. Does anyone know what this could be?

 

I've tried deleting the query and rebuilding it and I've tried deleting the entire dataflow and rebuilding that as well to no avail. 

8 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi paulenomial ,

    Thanks for using Fabric Community. This might be an intermittent issue, can you please re try it again?
    Please let me know incase of further queries.

    • paulenomial's avatar
      paulenomial
      Frequent Visitor

      Hi Anonymous ,

       

      I'm afraid I've tried multiple times over the last few days and it doesn't appear to be going away.

       

      Thanks,
      Paul.

  • pqian_MSFT's avatar
    pqian_MSFT
    Icon for Microsoft Employee rankMicrosoft Employee

    I see that the query didn't fold and it attempted to buffer all the rows locally. 

     

    can you right click on the final entity and select "view query plan", what do you see there?

    • paulenomial's avatar
      paulenomial
      Frequent Visitor

      Hi pqian_MSFT ,

       

      The green indicator on the rows indicates that it is folding however the final step takes approximately 1m 30s to load but the step prior takes only about 20s despite both steps being a concatenation of two fields.

       

      See query plan below along with the native data source query:

      select [_].[costamountadjustment] as [costamountadjustment],
          [_].[costamountoperations] as [costamountoperations],
          [_].[costamountphysical] as [costamountphysical],
          [_].[costamountposted] as [costamountposted],
          [_].[dateexpected] as [dateexpected],
          [_].[datefinancial] as [datefinancial],
          [_].[datephysical] as [datephysical],
          [_].[inventdimid] as [inventdimid],
          [_].[inventtransorigin] as [inventtransorigin],
          [_].[invoiceid] as [invoiceid],
          [_].[itemid] as [itemid],
          [_].[packingslipid] as [packingslipid],
          [_].[qty] as [qty],
          [_].[voucher] as [voucher],
          [_].[voucherphysical] as [voucherphysical],
          [_].[dataareaid] as [dataareaid],
          [_].[referenceid] as [referenceid],
          [_].[referencecategory] as [referencecategory],
          [_].[configid] as [configid],
          [_].[inventbatchid] as [inventbatchid],
          [_].[inventcolorid] as [inventcolorid],
          [_].[inventlocationid] as [inventlocationid],
          [_].[inventsiteid] as [inventsiteid],
          [_].[inventsizeid] as [inventsizeid],
          [_].[inventstatusid] as [inventstatusid],
          [_].[inventstyleid] as [inventstyleid],
          [_].[wmslocationid] as [wmslocationid],
          convert(nvarchar(4000), [_].[_FKinventlocation]) as [_FKinventlocation],
          convert(nvarchar(4000), [_].[_FKinventsite]) as [_FKinventsite],
          convert(nvarchar(4000), [_].[_FKinventtable]) as [_FKinventtable]
      from 
      (
          select [_].[costamountadjustment] as [costamountadjustment],
              [_].[costamountoperations] as [costamountoperations],
              [_].[costamountphysical] as [costamountphysical],
              [_].[costamountposted] as [costamountposted],
              [_].[dateexpected] as [dateexpected],
              [_].[datefinancial] as [datefinancial],
              [_].[datephysical] as [datephysical],
              [_].[inventdimid] as [inventdimid],
              [_].[inventtransorigin] as [inventtransorigin],
              [_].[invoiceid] as [invoiceid],
              [_].[itemid] as [itemid],
              [_].[packingslipid] as [packingslipid],
              [_].[qty] as [qty],
              [_].[voucher] as [voucher],
              [_].[voucherphysical] as [voucherphysical],
              [_].[dataareaid] as [dataareaid],
              [_].[referenceid] as [referenceid],
              [_].[referencecategory] as [referencecategory],
              [_].[configid] as [configid],
              [_].[inventbatchid] as [inventbatchid],
              [_].[inventcolorid] as [inventcolorid],
              [_].[inventlocationid] as [inventlocationid],
              [_].[inventsiteid] as [inventsiteid],
              [_].[inventsizeid] as [inventsizeid],
              [_].[inventstatusid] as [inventstatusid],
              [_].[inventstyleid] as [inventstyleid],
              [_].[wmslocationid] as [wmslocationid],
              [_].[dataareaid] + [_].[inventlocationid] as [_FKinventlocation],
              [_].[dataareaid] + [_].[inventsiteid] as [_FKinventsite],
              [_].[dataareaid] + [_].[itemid] as [_FKinventtable]
          from 
          (
              select [$Outer].[costamountadjustment],
                  [$Outer].[costamountoperations],
                  [$Outer].[costamountphysical],
                  [$Outer].[costamountposted],
                  [$Outer].[dateexpected],
                  [$Outer].[datefinancial],
                  [$Outer].[datephysical],
                  [$Outer].[inventdimid],
                  [$Outer].[inventtransorigin],
                  [$Outer].[invoiceid],
                  [$Outer].[itemid],
                  [$Outer].[packingslipid],
                  [$Outer].[qty],
                  [$Outer].[voucher],
                  [$Outer].[voucherphysical],
                  [$Outer].[dataareaid],
                  [$Outer].[referenceid],
                  [$Outer].[referencecategory],
                  [$Inner].[configid],
                  [$Inner].[inventbatchid],
                  [$Inner].[inventcolorid],
                  [$Inner].[inventlocationid],
                  [$Inner].[inventsiteid],
                  [$Inner].[inventsizeid],
                  [$Inner].[inventstatusid],
                  [$Inner].[inventstyleid],
                  [$Inner].[wmslocationid]
              from 
              (
                  select [$Outer].[costamountadjustment],
                      [$Outer].[costamountoperations],
                      [$Outer].[costamountphysical],
                      [$Outer].[costamountposted],
                      [$Outer].[dateexpected],
                      [$Outer].[datefinancial],
                      [$Outer].[datephysical],
                      [$Outer].[inventdimid],
                      [$Outer].[inventtransorigin],
                      [$Outer].[invoiceid],
                      [$Outer].[itemid],
                      [$Outer].[packingslipid],
                      [$Outer].[qty],
                      [$Outer].[voucher],
                      [$Outer].[voucherphysical],
                      [$Outer].[dataareaid],
                      [$Inner].[referenceid],
                      [$Inner].[referencecategory]
                  from 
                  (
                      select [costamountadjustment],
                          [costamountoperations],
                          [costamountphysical],
                          [costamountposted],
                          [dateexpected],
                          [datefinancial],
                          [datephysical],
                          [inventdimid],
                          [inventtransorigin],
                          [invoiceid],
                          [itemid],
                          [packingslipid],
                          [qty],
                          [voucher],
                          [voucherphysical],
                          [dataareaid]
                      from [dataverse_production_cds2_workspace_unq239aaf6e46184884942eb81311b62].[dbo].[inventtrans] as [$Table]
                  ) as [$Outer]
                  left outer join 
                  (
                      select [$Outer].[inventtransid] as [inventtransid],
                          [$Outer].[referenceid] as [referenceid],
                          [$Outer].[dataareaid] as [dataareaid2],
                          [$Outer].[recid] as [recid],
                          [$Inner].[LocalizedLabel] as [referencecategory]
                      from 
                      (
                          select [referencecategory],
                              [inventtransid],
                              [referenceid],
                              [dataareaid],
                              [recid]
                          from [dataverse_production_cds2_workspace_unq239aaf6e46184884942eb81311b62].[dbo].[inventtransorigin] as [$Table]
                      ) as [$Outer]
                      left outer join 
                      (
                          select [_].[Option],
                              [_].[LocalizedLabel],
                              [_].[ExternalValue]
                          from [dataverse_production_cds2_workspace_unq239aaf6e46184884942eb81311b62].[dbo].[GlobalOptionsetMetadata] as [_]
                          where ([_].[EntityName] = 'inventtransorigin' and [_].[EntityName] is not null) and ([_].[OptionSetName] = 'referencecategory' and [_].[OptionSetName] is not null)
                      ) as [$Inner] on ([$Outer].[referencecategory] = [$Inner].[Option] or [$Outer].[referencecategory] is null and [$Inner].[Option] is null)
                  ) as [$Inner] on ([$Outer].[inventtransorigin] = [$Inner].[recid] or [$Outer].[inventtransorigin] is null and [$Inner].[recid] is null)
              ) as [$Outer]
              left outer join 
              (
                  select [configid] as [configid],
                      [inventbatchid] as [inventbatchid],
                      [inventcolorid] as [inventcolorid],
                      [inventdimid] as [inventdimid2],
                      [inventlocationid] as [inventlocationid],
                      [inventsiteid] as [inventsiteid],
                      [inventsizeid] as [inventsizeid],
                      [inventstatusid] as [inventstatusid],
                      [inventstyleid] as [inventstyleid],
                      [wmslocationid] as [wmslocationid],
                      [dataareaid] as [dataareaid2]
                  from [dataverse_production_cds2_workspace_unq239aaf6e46184884942eb81311b62].[dbo].[inventdim] as [$Table]
              ) as [$Inner] on (([$Outer].[inventdimid] = [$Inner].[inventdimid2] or [$Outer].[inventdimid] is null and [$Inner].[inventdimid2] is null) and ([$Outer].[dataareaid] = [$Inner].[dataareaid2] or [$Outer].[dataareaid] is null and [$Inner].[dataareaid2] is null))
          ) as [_]
      ) as [_]

       

      • pqian_MSFT's avatar
        pqian_MSFT
        Icon for Microsoft Employee rankMicrosoft Employee

        This query plan should not be buffering at all.

        What is the entity name? The one causing issue is called "inventtrans".