Importing Newly Added JSONs from Folder Source

[Tkre]

I'm trying to use the folder sourcing for a constantly expanding library of JSON files that we use for production tracking.  I'm trying to seperate out only the date/time portion from "ScanHistory" and combine that from all files to be able to model production from the raw data.  I have no trouble setting up the query to do that by transposing, merging, and splitting but when new files are added they aren't included with the merge step.  I have a feeling that the sample file paramenters are key to pre-parsing the files to end up with a single column from the combined data that I don't have to transpose and merge but I'm not familiar enough with it to know if that's true or how to do so.  

I've included a small sample file to show what I'm dealing with.

{"ScanHistory": "[('Scanner', '744211-04', 1, '12/15/18 07:28:00:095000'), ('Scanner', '744211-04', 1, '12/15/18 07:28:11:373000'), ('Scanner', '744211-04', 1, '12/15/18 07:28:13:488000')]", "ShiftNotes": "", "Operators": {"1": "Sharon", "2": "Laura"}, "Shift": "1", "Date": "12/15/2018", "Work Order": 5118, "Counters": {"Inventory ASM": 0, "Cycle Time": 0, "NC Out Per": 0, "Boxes Made": 0, "NC ASM": 0, "Parts Made": 3, "Start Parts Original": 0, "Unchanged Original Start": 0, "Cleared ASM": 0, "QC ASM": 0, "Parts Remaining at Start": 880, "Pallets Made": 0, "IN Table": {"Light Symbol": {"QCheck": 0, "NC": 0, "QCleared": 0}, "Overlay": {"QCheck": 0, "NC": 0, "QCleared": 0}, "Rubber": {"QCheck": 0, "NC": 0, "QCleared": 0}, "Tape": {"QCheck": 0, "NC": 0, "QCleared": 0}, "Substrate": {"QCheck": 0, "NC": 0, "QCleared": 0}, "Electronics": {"QCheck": 0, "NC": 0, "QCleared": 0}, "Light Guide": {"QCheck": 0, "NC": 0, "QCleared": 0}}, "Start Parts": 0, "NC In Count": 0, "Good ASM": 3, "Total Production": 3}}

[v-juanli-msft]

Hi @Tkre 

Do you connect  to folders with Power BI Desktop as this article "How to Load Data from a Folder in Power BI"?

Does the new file have the same structure as previous ones?

Best Regards

Maggie

[Tkre]

I follow that but have slightly different options since there's no combine and load.  When I click combine and edit, it goes straight to the editor without a preview either.

In the editor I then get this:

I've then been filtering the rows to get just "ScanHistory" which has the entire scan time history for that work order mixed with the barcode data in each row.  I was using transpose then merge to get all of them into a single cell then split using delimiters to break it out into a single long column and filtering out non-dates until I had only the date/time left.  The problem was merge only selects the actual columns present during the step so every file added would add a column that wouldn't be included in that step.

Worst case I have the production software output into a seperate .csv in addition to the production files but I've really like to directly read the production data if possible to have a single source of truth.