Forum Discussion

JoeCrozier's avatar
JoeCrozier
Icon for Helper II rankHelper II
1 year ago
Solved

Dealing with "record" columns in a Dataflow gen2

I have this dataflow that pulls from a Snowflake destination:

As you can see in the screenshot, a few of the columns are of "record" type.  Fabric doesnt seem to like these formats and automatically deletes them sometimes by adding code that looks like this:

 

Table.RemoveColumns(#"From Value", Table.ColumnsOfType(#"From Value", {type table, type record, type list, type nullable binary, type binary, type function}))


Makes sense, as best as I can understand it the data needs to be flat, almost csv like.

So the way I understand it, I need to expand out those records, maybe in their own table.  So I copied that query and selected just two columns:


When I tell Fabric to expand that contact identifier column and grab every column inside of it:


it doesn't "really" expand everything.  Here's what I mean, here's whats visible immediately after that:

 

That "step is not supported by fast copy" and still wont be able to save into a Lakehouse.  What do I do?

  • lbendlin's avatar
    lbendlin
    1 year ago

    filter out all null value, then expand the record column again.

3 Replies

  • looks like this might be a record inside a record. See if you can grab "value"  instead.

      • lbendlin's avatar
        lbendlin
        Icon for Super User rankSuper User

        filter out all null value, then expand the record column again.