Forum Discussion
Progrmatically write files in delta
I was able to use this code to write to a simple text file:
import os # Define the sentence you want to write sentence = "This is the sentence that will be written to the text file." # Specify the folder path and file name folder_base_path = "/lakehouse/default/Files/" folder_relative_path = "sentence_files" file_name = "output.txt" folder_path = os.path.join(folder_base_path, folder_relative_path) # Combine the folder path and file name file_path = os.path.join(folder_path, file_name) # Create the directory if it doesn't exist os.makedirs(folder_path, exist_ok=True) # Open the file in write mode and write the sentence with open(file_path, "w") as file: file.write(sentence) print(f"Sentence written to {file_path}")The folder_base_path will depend on whether your notebook has a default lakehouse or if you are just mounting lakehouses to your notebook.
In the code I show above, the folder_base_path assumes that the notebook has a default lakehouse.
If you don't want to use a default lakehouse, then you will need to mount a lakehouse instead. However if you don't have any specific requirements, I would say just use a default lakehouse for your notebook.
https://fabric.guru/how-to-mount-a-lakehouse-and-identify-the-mounted-lakehouse-in-fabric-notebook
By default, PySpark creates a folder with multiple files. I guess this is because PySpark uses distributed processing on multiple worker nodes.
If you want to write a (not too big) dataframe to a single file, I think the easiest way is to use Pandas.
"The file name control is an important aspect of my workflow, hence I cant stick writing to files using codes in notebook. I don't know if there is a lakehouse api that lets you write(upload) a file with developer created content such that the file name could be exactly same as dev desired."
This can be done with similar code like the one I used in the previous comment.
It can also be done with Pandas.
I think also the ADLS Gen2 API can be used, it supposedly works with OneLake. Here is an example of someone who has used the API to connect to OneLake from Power Automate. I guess you can use the API from any client, not just from Power Automate. https://www.linkedin.com/pulse/how-call-onelake-api-from-power-automate-enterprise-app-nigel-smith-4szoc?utm_source=share&utm_medium=guest_mobile_web&utm_campaign=copy
PowerShell might also be an option:
https://learn.microsoft.com/en-us/fabric/onelake/onelake-powershell
There is also the OneLake explorer, where we can interact with OneLake as a folder structure on our local machine.
frithjof_v it worked. Many thanks for this. I just tried out. This is exactly what I had in my mind.
