Forum Discussion
Gen2 fails while Gen1 works
We were finally able to determine what the issue is and we have a fix coming in the next few weeks.
The root cause of the issue is a bug where the function still has the "staging" setting to be enabled regardless if the UI is not showing it. You can verify this by going into the GIT of the Dataflow and seeing the json file for the metadata of the queries in your Dataflow.A simple workaround is to either start with a blank query that doesn't have the staging enabled OR transform the function to a query, change the staging to be disabled and then transform your query back to a function.
miguel follow this to reproduce
#1 create a function
//fnGetTimestamp
() =>
let
Source = Sql.Database("abc.datamart.fabric.microsoft.com","def"),
NativeResult = Value.NativeQuery(Source, "SELECT GETDATE() AS timestamp", null, [EnableFolding = true]),
TimestampValue = Record.Field(NativeResult{0}, "timestamp")
in
TimestampValue
#2. create a sharepoint query and invoke above on that query and try to insert into a lakehouse destination
#3. It failed for me at #2
what actually worked
#1 I was copying this function from my github repo as I have them for quick copy paste
## intead of copy paste, I ran the query as below and chose to convert to function from the context menu
let
Source = Sql.Database("abc.datamart.fabric.microsoft.com","def"),
NativeResult = Value.NativeQuery(Source, "SELECT GETDATE() AS timestamp", null, [EnableFolding = true]),
TimestampValue = Record.Field(NativeResult{0}, "timestamp")
in
TimestampValue
## once PQ converted the above into a function with blank parameters, I added parameters as required and adjusted the function.
## it also creates a grouping
## from this point onwards I had no issue invoking into other queries
#2. create a sharepoint query and invoke above on that query and try to insert into a lakehouse destination
#3. It failed for me at #2Thank you for bringing this up! I was able to repro the issue and I'm engaging internally with my engineering team. Tough to say why this is behaving this way, but we'll definitely look into it.
Thanks again and please keep the feedback coming!