Forum Discussion

joseanmarsol's avatar
joseanmarsol
New Member
2 years ago
Solved

Build array columns in a dataflow

Dear community,    I am trying to move data pipelines from Azure Data Factory (ADF) to Fabric Data Factory (FDF).   I am inputting exchange rates data in the form of xml adn I need to output as J...
  • ImkeF's avatar
    2 years ago

    Hi joseanmarsol ,
    if my understanding is correct, you need the JSON as a textual representation in a column in dataflow to use it in a later pipeline action to make that API call?
    Then the solution from Anonymous  won't work, as it returns a binary that cannot be loaded properly into a lakehouse table.
    But with a small adjustment, you will get it as needed:

    let
        Source = ExchangeRates,
        #"Changed Type" = Table.TransformColumnTypes(Source,{{"date", type date}}),
        #"Grouped Rows" = Table.Group(#"Changed Type", {"provider", "date", "baseCurrency"}, {
            {"exchangeRates", each Table.SelectColumns(_, {"currency", "rate"})}
        }),
        #"Added Custom" = Table.AddColumn(#"Grouped Rows", "JSON", each Text.FromBinary(Json.FromValue([exchangeRates]))),
        #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"exchangeRates"})
    in
        #"Removed Columns"

    I don't know about the size limits for fields in fabric lakehouse, but if you have really large tables, you might want to check for that.