Forum Discussion

pwr20's avatar
pwr20
Regular Visitor
4 months ago
Solved

Dataflow Gen2 File Destination - JSON and Parquet Format Support Needed for Nested OData Data

I'm integrating a third party OData API hosted on an on-premises server accessible only via on-premises data gateway. Current Situation: OData source has deeply nested columns Dataflow Gen2 can c...
  • v-prasare's avatar
    4 months ago

    Hi pwr20 , 

    Currently, Microsoft Fabric Dataflow Gen2 supports only CSV/delimited formats for file-based destinations such as Lakehouse Files and ADLS Gen2. These formats do not support nested data structures, which makes them unsuitable for landing raw nested OData data without flattening.

    When working with nested OData navigation properties, Dataflow Gen2 treats nested records as unsupported for tabular or file destinations. As a result, nested columns are either dropped or replaced with null values when writing to Lakehouse tables, staging destinations, or files. Fully expanding nested data within Dataflow can work for small volumes but does not scale reliably, especially when using an on‑premises data gateway, due to sequential API calls and timeout limitations.

    At this time, JSON or Parquet file destinations are not available in Dataflow Gen2, and there is no built‑in, scalable solution to preserve nested OData structures end‑to‑end. This is a current product limitation, not a configuration issue.

     

    I’d encourage you to submit your detailed feedback and ideas via Microsoft's official feedback channels, such as the Microsoft Fabric Ideas.

    Feedback submitted here is often reviewed by the product teams and can lead to meaningful improvement.

     

     

     

     

    Thanks,

    Prashanth