Forum Discussion

abhidotnet's avatar
abhidotnet
Advocate II
2 days ago

How do I unzip a .gz file?

I used a data pipeline to make a web call (http) and get a file. 
The file has been downloaded to the Files area in the lakehouse. How can I uncompress this file? 

I am using a PySpark notebook to unzip this file. The file itself is good, since I was able to download the file and uncompress it on my windows machine. 

I tried this code, but that fails.

df = spark.read.format("json").option("multiLine", "false").load("Files/bronze/github-events-2025-01-15-12.json.gz")
# Display a preview
display(df.limit(5))
# Save as Delta table
df.write.mode("overwrite").format("delta").saveAsTable("github_events_bronze")
print(f"Successfully loaded {df.count()} records")
data pipeline
lakehouse

2 Replies

  • sandeephijam's avatar
    sandeephijam
    Helper II
    1 day ago

     Hi abhidotnet​ ,

    Hi Abhi please see if you share the error message to understand the failure,

    As an immediate check you can do the following to check 

    File existance  : display(mssparkutils.fs.ls("Files"))  and  then display(mssparkutils.fs.ls("Files/bronze"))

    Also check if we have attached the lakehouse

  • tayloramy's avatar
    tayloramy
    Super User
    1 day ago

    Hi abhidotnet​,

    You can use the gzip package in Python to unzip a .gz file. 

    import gzip
    import shutil

    # Define paths (e.g., in your Lakehouse Files section)
    input_path = "/lakehouse/default/Files/my_data.csv.gz"
    output_path = "/lakehouse/default/Files/my_data.csv"

    # Decompress the .gz file
    with gzip.open(input_path, "rb") as f_in:
            with open(output_path, "wb") as f_out:
                    shutil.copyfileobj(f_in, f_out)
    print(f"Successfully unzipped to {output_path}")