Forum Discussion

VoltesDev's avatar
VoltesDev
Helper V
2 years ago

How to merge multiple JSON files into one

Hi,

 

I have 2 JSON file with the same structure. May I know what is best to merge it ? 

These is just a JSON file from "bing seach" result, only with different market.

Are we doing it in data pipeline ? or we doing it in Notebook ? and because I'm still learning, may I know how to do it to be precise ?

 

Thanks

Tutorial Requests

3 Replies

  • tharunkumarRTK's avatar
    tharunkumarRTK
    Super User
    2 years ago

    VoltesDev 

    Assuming you have two json files with same structure and you want to parse and append them as a table in Fabric. There are multiple ways to do this task, one of the qucik and easy way is through dataflow (gen 1 or gen 2, depending on your usecase). 

     

    Move these two josn files into a Lakehouse folder and create a new dataflow and connect to the lakehouse and combine the files.

     

    You can follow these intructions: https://www.youtube.com/watch?v=Nbhd0B5ldJE

     



    Need Power BI consultation, hire me on UpWork .


    If the post helps please give a thumbs up



    If it solves your issue, please accept it as the solution to help the other members find it more quickly.




    Tharun



     

    • VoltesDev's avatar
      VoltesDev
      Helper V
      2 years ago

      Hi tharunkumar,

       

      But the video looks like we need to merge it from PowerBI Desktop, doesn't it ? by using Power Query ?

      Can we do that inside Fabric ? 

       

      Can we do it in the pipeline ? is there merge feature in there ? If I look into Data Factory, there is Union feature, but can't find in Fabric.  I was also thinking probably need to merge using notebook, and merge in within using Python code.

       

      Thanks

       

       

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi, VoltesDev 

    Thanks for tharunkumarRTK reply. You can try following method to merge multiple JSON into one.

    1. Use Python to read the JSON files, extract the relevant data, and merge them into a single file.

    import json

# Create a list of all the JSON files that you want to combine.
json_files = ["file1.json", "file2.json", "file3.json"]

# Create an empty list to store the Python objects.
python_objects = []

# Load each JSON file into a Python object.
for json_file in json_files:
    with open(json_file, "r") as f:
        python_objects.append(json.load(f))

# Dump all the Python objects into a single JSON file.
with open("combined.json", "w") as f:
    json.dump(python_objects, f, indent=4)

    2. Use the jq command-line tool

    jq -s . file1.json file2.json file3.json > combined.json


    How to Combine Multiple JSON Files into a Single JSON File | by Abdelfatah MENNOUN | Medium
    python 3.x - Merge multiple JSON files (more than two) - Stack Overflow

    Best Regards,
    Yang
    Community Support Team

     

    If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
    If I misunderstand your needs or you still have problems on it, please feel free to let us know.     Thanks a lot!

    How to get your questions answered quickly --  How to provide sample data in the Power BI Forum