Forum Discussion
MashupException.Error while trying to load incremental data by DateTime field
Hey there! I am the PM that wrote the guide on how to do this. This is a great way to amass your data incrementally and using the date field makes sense!
What I noticed is that you still have the first query that retrieves the timestamp with staging enabled. Can you try to disable the staging on the query that retrieves the timestamp?
additionally I noticed you used the the first row from the output as getting the latest timestamp. Could you consider using this function instead? It is a bit simpler and cleaner.
Thank you LuitwielerMSFT , your suggestion turns out to resolve the issue. Would you bother to briefly explain how "Mashup" and "Staging" are correlated? I guess with disabling "Staging" I circumvent the need for the Mashup to happen, which seem to not support that kind of operation?
Or has it been the way I retrieved the maximum Date for further processing?
Awesome! Well what happens is that when staging is turned on, the dataflow engine is storing the results into the staging lakehouse/warehouse. When the dataflow engine then is asked to use the results of the query it retrieves it from the "staging table" and makes the result a "table variable". I'm taking this feedback with me to the engineers while we work on a better design for staging and how we work with variables stored in staging that are not in a table format. Thanks for highlighting this! And stay tuned for the incremental refresh feature that is on the roadmap 😉
Thanks for the details LuitwielerMSFT
Glad that your query got resolved Anonymous . Please continue using Fabric Community for any help regarding your queries.