Lost Notebook Changes in Synapse Data Science (Microsoft Fabric)
Hello Fabric Community, I recently encountered an unexpected issue with my notebook in Synapse Data Science (Microsoft Fabric). Despite having AutoSave enabled, I lost all the changes I made over the past few days. Strangely, only a few code snippets from five days ago remain. 🤔 Additionally, I couldn’t find a history tab for my notebook within Fabric. Has anyone else faced a similar problem? If you’ve found a solution or have insights to share, please join the discussion.
Help with notebooks / multiple .csv files for complete newbie
Hi, I have a need to ingest a ton of Power BI audit logs into Fabric. The eventual target is a Lakehouse table or tables. I have one log file per day (as csv), about 500 altogether. These files need extensive reshaping - things like pivoting attribute:value pairs into columns, parsing the text, etc. There's also reasonably large - about 40 or 50 million total lines, and maybe 4 or 5 million distinct audit log entries (see image below of a single entry) I was going to do this using a Gen2 dataflow, but I can't figure out how to get a Gen2 dataflow to ingest a folder of files. Works perfectly with our on-prem network file shares, but in Fabric the Gen2 dataflow seems to want only one file at a time, no way to combine things under a Lakehouse Files section. I'm a complete newbie on notebooks and really barely understand any Python (working to rectify that, going to take some time). Here's a sample log file entry: Things I've tried: 1. Using some of the sample code in the "learn fabric" paths to combine the files and publish to a Delta lakehouse table. This doesn't really work - if set VORDER on it screws up the order of lines in the audit log, making it impossible to associate lines with the proper audit log ID. If I turn VORDER off, querying the resulting table returns "Failed to read parquet file because the column segment for column '_c0' is too large" 2. I've tried using code like the following to append the 500 individual csv files into a single file. But it doesn't work - it seems instead to create a folder that has a ton of separate files in it instead of a single large file df = spark.read.format("csv").load('Files/Power BI audit logs/Audit log*.csv') df.write.mode("overwrite").format("csv").save("Files/singleFile") Any ideas or guidance would be greatly appreciated - thanks! Scott
