Download sharepoint files to lakehouse

[DiKi-I]

Hi,
I'm able to read the files and folders withing sharepoint using graphapi, how I can write those file into my lakehouse, can someone please help?

import requests

# Replace with your actual values

tenant_id = ""

client_id = ""

client_secret = ""

sharepoint_url = "xxxx.sharepoint.com/"

site_name = "ADFPOC"

# Step 1: Get OAuth2 token

token_url = f"https://login.microsoftonline.com/{tenant_id}/oauth2/token"

token_data = {

    "grant_type": "client_credentials",

    "client_id": client_id,

    "client_secret": client_secret,

    "resource": "https://graph.microsoft.com"

}

response = requests.post(token_url, data=token_data)

response.raise_for_status()

access_token = response.json()["access_token"]

# Step 2: Test SharePoint Site ID lookup

headers = {

    "Authorization": f"Bearer {access_token}"

}

site_info_url = f"https://graph.microsoft.com/v1.0/sites/{sharepoint_url}:/sites/{site_name}?$select=id"

response = requests.get(site_info_url, headers=headers)

response.raise_for_status()

site_id = response.json()["id"]

print("✅ Successfully connected to SharePoint.")

print(f"Site ID: {site_id}")


------------------------------------------------------------

[DiKi-I · Accepted solution]

Thanks I was able to directly download the file to lakehouse using fabric notebook without converting it into dataframe. #solved

[ibarrau]

Hi. You probably could do it from a Fabric Notebook. Get the data, put it on a spark dataframe and then store it at lakehouse as csv, parquet or delta. When working at a fabric notebook you can pick a lakehouse to make it easy to read and write. You can check it here for examples: https://learn.microsoft.com/en-us/fabric/data-engineering/lakehouse-notebook-load-data

On the other hand, that would be way easier if you use a Dataflow gen2 item at Fabric to connect the sharepoint, pick the file or files and configure the lakehouse destination. The UI is very friendly.

I hope that helps,