Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Join us at the 2025 Microsoft Fabric Community Conference. March 31 - April 2, Las Vegas, Nevada. Use code FABINSIDER for $400 discount. Register now

Reply
TimoRiikonen
Helper V
Helper V

Dataflow::Can't save content to Lakehouse if I filter it first with date

I wish to:

1) Copy data from Lakehouse bronze layer to the same Lakehouse silver layer

2) Remove extra columns

3) Only copy today's and yesterday's data

 

First two steps work fine, but when I add the third step, then I get an error.

Silver table:

TimoRiikonen_0-1713783818160.png

Code to create yesterday's date:

 

 

 

 

Date.AddDays( Date.From( DateTime.LocalNow() ), -1)

 

 

 

 

Result is 4/21/2024 (or 4/12/2024), so the date is in US-en format, same as in Lakehouse.

 

 Code to do the data copying:

 

 

 

 

let
  Source = Lakehouse.Contents([]),
  #"Navigation 1" = Source{[workspaceId = "8466c1bc-e477-415e-8b77-a714cd043631"]}[Data],
  #"Navigation 2" = #"Navigation 1"{[lakehouseId = "f236f3df-1701-4605-842d-415b6cb3ba37"]}[Data],
  #"Navigation 3" = #"Navigation 2"{[Id = "ServiceRequest", ItemKind = "Table"]}[Data],
  #"Removed other columns" = Table.SelectColumns(#"Navigation 3", {"Tilaus_id", "Pyynnön_alatyyppi", "Pyyntötyyppi", "Pyyntötyyppi_id", "Tilaus", "Updated"}),
  #"Changed column type" = Table.TransformColumnTypes(#"Removed other columns", {{"Updated", type date}}),
  #"Filtered rows" = Table.SelectRows(#"Changed column type", each [Updated] >= MinUpdateDate)
in
  #"Filtered rows"

 

 

 

 

When I refresh my data, everything works still fine:

TimoRiikonen_1-1713784079731.png

Publish over new or existing, same result:

TimoRiikonen_2-1713784181804.png

After I publish it, I get this error:

TimoRiikonen_3-1713784329206.png

 

 

 

 

ServiceRequest_WriteToDataDestination: Mashup Exception Expression Error Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Expression.Error: We cannot apply operator < to types Table and Date. Details: Reason = Expression.Error;Operator = <;Left = error "Microsoft.Mashup.Engine1.Runtime.ValueException: [Expression.Error] Value was not specified.#(cr)#(lf)   at Microsoft.Mashup.Engine1.Language.ValueCreator.CreateValueForThrow(IThrowExpression throwExpr)#(cr)#(lf)   at Microsoft.Mashup.Engine1.Language.ValueCreator.<>c__DisplayClass23_0.<CreateValueForRecord>b__0(Int32 index)#(cr)#(lf)   at Microsoft.Mashup.Engine1.Runtime.RecordValue.DemandRecordValue.get_Item(Int32 index)#(cr)#(lf)   at Microsoft.Data.Mashup.ProviderCommon.MashupResource.TryGetValue(Func`1 getValue, IValue& value, String& errorMessage)#(cr)#(lf)Record";Right = 4/17/2024 12:00:00 AM;Microsoft.Data.Mashup.Error.Context = User

 

 

 

 

 

If I remove the last filter, then everything works as it should (I had -10, not -1 so ignore the date):

TimoRiikonen_4-1713784998499.png

 

 

1 ACCEPTED SOLUTION

Upon further inspection of the error, my suspicion is correct. The query MinUpdateDate is more than likely set to be staged which causes the result of the query to be set to a table. Then you're trying to use that table value as a filter to a column that expects a date and that's why the error displays that specific error message.

The solution is to disable the staging from the MinUpdateDate query.

View solution in original post

5 REPLIES 5
TimoRiikonen
Helper V
Helper V

I tested filtering on ID field and there was no issues then.

miguel
Community Admin
Community Admin

Is your query that calculate the date set to be staged? If yes, please disable staging for it and try running the dataflow again 

 

Upon further inspection of the error, my suspicion is correct. The query MinUpdateDate is more than likely set to be staged which causes the result of the query to be set to a table. Then you're trying to use that table value as a filter to a column that expects a date and that's why the error displays that specific error message.

The solution is to disable the staging from the MinUpdateDate query.

Hello Miguel,

 

You were correct.

Table was not staged, but the parameter was.

After removing this, it is working again.

 

Thank you!

 

Hi @TimoRiikonen 

 

Glad that you query got resolved.
Please continue using Fabric Community for further queries.

 

Thanks.

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!

FebFBC_Carousel

Fabric Monthly Update - February 2025

Check out the February 2025 Fabric update to learn about new features.

Feb2025 NL Carousel

Fabric Community Update - February 2025

Find out what's new and trending in the Fabric community.

Top Solution Authors