Forum Discussion

ayyappa5678220's avatar
ayyappa5678220
New Member
4 months ago
Solved

Data ingesting from API to Lakehouse, file format wrong, how to resolve?

Hi Team, I am new to MS Fabric, Here I am ingesting data from API(git) to lakehouse using copy data activity in fabric pipeline. And data is in format of Zip file in API.  And pipeline running succ...
data engineering
data pipeline
lakehouse
notebook
  • nilendraFabric's avatar
    nilendraFabric
    4 months ago

    hi ayyappa5678220 

    This is a common problem when copying ZIP files via pipeline without extracting them, or when the file lands without a proper extension.

    give this a try 

    In your Fabric Pipeline, after the Copy Data activity, add a Notebook activity to unzip and load the files:

    import zipfile, os, shutil

    src="/lakehouse/default/Files/data/efa0b651-c7d4-4282-92fc-21c3e11464a2"
    dest = "/lakehouse/default/Files/data/extracted/"

    os.makedirs(dest, exist_ok=True)

    if zipfile.is_zipfile(src):
    with zipfile.ZipFile(src, 'r') as z:
    z.extractall(dest)
    print("Extracted:", z.namelist())






nilendraFabric's avatar
nilendraFabric
Super User
4 months ago

hi ayyappa5678220 

This is a common problem when copying ZIP files via pipeline without extracting them, or when the file lands without a proper extension.

give this a try 

In your Fabric Pipeline, after the Copy Data activity, add a Notebook activity to unzip and load the files:

import zipfile, os, shutil

src="/lakehouse/default/Files/data/efa0b651-c7d4-4282-92fc-21c3e11464a2"
dest = "/lakehouse/default/Files/data/extracted/"

os.makedirs(dest, exist_ok=True)

if zipfile.is_zipfile(src):
with zipfile.ZipFile(src, 'r') as z:
z.extractall(dest)
print("Extracted:", z.namelist())