apparent bug re custom functions and gen2 ci/cd dataflows

[otis_pc]

Hi, all.

I regularly use custom function queries that I then call in other queries. I have spent that last few hours determining that apparently the ci/cd dataflows don't recognize custom functions if they're external to the query you're working in.

For example, the following would usually be its own query, which takes in a string and a predefined table containing columns "find" and "replace". This method works in gen1 and regular gen2 dataflows:

// query named fxReplaceMultipleText
    (txt as text, tbl as table) as text => 
      let
        cnt=Table.RowCount(tbl)
        , ret=
            List.Accumulate(
              List.Numbers(0,cnt)
              , txt
              , (state, current) =>
                Text.Replace(state, tbl{current}[find], tbl{current}[replace])
            )
      in ret

and I would invoke it like so:

let
  Source = srcTbl,
  valueTbl = Table.Buffer(TagValueSymbolReplacements & ALL_TagKeyReplacements),
  keyTbl = Table.Buffer(ALL_TagKeySymbolReplacements & ALL_TagKeyReplacements),
  #"Choose columns" = Table.SelectColumns(Source, {"Index", "Date", "ResourceGroup", "Tags"}),
  splitByComma = Table.ExpandListColumn(Table.TransformColumns(#"Choose columns", {{"Tags", Splitter.SplitTextByDelimiter(","), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "Tags"),
  splitByColon = Table.SplitColumn(splitByComma, "Tags", Splitter.SplitTextByEachDelimiter({":"}, QuoteStyle.Csv), {"key", "value"}),
  replaceKey = Table.ReplaceValue(splitByColon, each [key], each fxReplaceMultipleText([key], keyTbl), Replacer.ReplaceValue, {"key"}),
  replaceValue = Table.ReplaceValue(replaceKey, each [value], each fxReplaceMultipleText([value], valueTbl), Replacer.ReplaceValue, {"value"}),
  #"Changed column type" = Table.TransformColumnTypes(replaceValue, {{"key", type text}, {"value", type text}})
in
  #"Changed column type"

 When it comes to gen2 ci/cd dataflows, the above method appears to work while you're IN the dataflow, but upon save and run

• if you enable loading the query, the refresh fails (with an error something like 'we ran into an error')
• if you load to a lakehouse, the dataflow refreshes but doesn't apply the replace steps.

Has anyone else run into this? For now I just have to define the custom function as a step in each query in which I invoke it. That rather defeats the purpose when you're calling it in multiple queries.