How to merge multiple JSON files into one

[VoltesDev]

Hi,

I have 2 JSON file with the same structure. May I know what is best to merge it ? 

These is just a JSON file from "bing seach" result, only with different market.

Are we doing it in data pipeline ? or we doing it in Notebook ? and because I'm still learning, may I know how to do it to be precise ?

Thanks

[v-yaningy-msft]

Hi, @VoltesDev 

Thanks for @tharunkumarRTK reply. You can try following method to merge multiple JSON into one.

1. Use Python to read the JSON files, extract the relevant data, and merge them into a single file.

import json

# Create a list of all the JSON files that you want to combine.
json_files = ["file1.json", "file2.json", "file3.json"]

# Create an empty list to store the Python objects.
python_objects = []

# Load each JSON file into a Python object.
for json_file in json_files:
    with open(json_file, "r") as f:
        python_objects.append(json.load(f))

# Dump all the Python objects into a single JSON file.
with open("combined.json", "w") as f:
    json.dump(python_objects, f, indent=4)

2. Use the jq command-line tool

jq -s . file1.json file2.json file3.json > combined.json

How to Combine Multiple JSON Files into a Single JSON File | by Abdelfatah MENNOUN | Medium
python 3.x - Merge multiple JSON files (more than two) - Stack Overflow

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

How to get your questions answered quickly --  How to provide sample data in the Power BI Forum

[tharunkumarRTK]

@VoltesDev 

Assuming you have two json files with same structure and you want to parse and append them as a table in Fabric. There are multiple ways to do this task, one of the qucik and easy way is through dataflow (gen 1 or gen 2, depending on your usecase). 

Move these two josn files into a Lakehouse folder and create a new dataflow and connect to the lakehouse and combine the files.

You can follow these intructions: https://www.youtube.com/watch?v=Nbhd0B5ldJE

Need Power BI consultation, hire me on UpWork .

If the post helps please give a thumbs up

If it solves your issue, please accept it as the solution to help the other members find it more quickly.

Tharun

[VoltesDev]

Hi tharunkumar,

But the video looks like we need to merge it from PowerBI Desktop, doesn't it ? by using Power Query ?

Can we do that inside Fabric ? 

Can we do it in the pipeline ? is there merge feature in there ? If I look into Data Factory, there is Union feature, but can't find in Fabric.  I was also thinking probably need to merge using notebook, and merge in within using Python code.

Thanks