Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
1 year ago
Solved

JSON issue with data lake

  I am currently using a pipeline to connect and bring in all of the tables i want from dataverse into my datalake. I use the method of copying from the data assistant, then connecting to data...
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi Anonymous,

    Thanks for reaching out to the Microsoft fabric community forum.

    The JSON format from a Copy Activity in Data Factory or Fabric Pipelines is not always a standard JSON array. Instead, it might output newline-delimited JSON (NDJSON), where each line is a separate JSON object.

    Ex:

    {"id":1,"name":"Alice"}
    {"id":2,"name":"Bob"}
    {"id":3,"name":"Charlie"}

    This is not valid as a JSON array (which looks like [{}, {}, {}]), and Power Query  which is used in Gen2 Dataflows expects valid JSON, not NDJSON. Maybe that why, when Power BI or Dataflow tries to read this as a single JSON file, it fails with the error "We found extra characters at the end of the JSON output", as it tries to parse the first line as a complete JSON, then hits the second object unexpectedly. If you want to use JSON, you need to convert the NDJSON into a proper JSON array before loading it into the Dataflow.

     

    If I misunderstand your needs or you still have problems on it, please feel free to let us know.  

    Best Regards,
    Hammad.
    Community Support Team

     

    If this post helps then please mark it as a solution, so that other members find it more quickly.

    Thank you.