Forum Discussion

ahmedshalabyy12's avatar
4 months ago
Solved

Copy paths

Hello , The 3rd option I don't understand it even with chat gpt, so I decided to get the answer from real expert human. First option ABFS Path : to use it if the lakehouse not pinne...
  • deborshi_nag's avatar
    4 months ago

    Hello ahmedshalabyy12 

     

    The third option is necessary when you want to access data using Python notebooks instead of PySpark. Spark APIs are not available in Python notebooks, and Pandas cannot interpret cloud protocols like ABFS. In these situations, you should use the third option, File API path, with pandas (read_parquet).

     

    Microsoft doc explicitly says the default Lakehouse is automatically mounted and that “Copy File API Path” returns the File API path from that mount point.

     

    Load data into your lakehouse with a notebook - Microsoft Fabric | Microsoft Learn

     

  • Asmita_27's avatar
    4 months ago

    Hello ahmedshalabyy12 ,

    Let me simplify all 3 options for you:

    1. ABFS Path (Full Path)
    • This is the complete address of your file in OneLake.
    • Example: abfss://[email protected]/lakehouse_id/Files/customers.parquet
    • Use it when the lakehouse is NOT pinned to your notebook, or when accessing data from a different lakehouse/workspace.
    • Works with both Spark and Pandas.

         2. Relative Path for Spark (Short Path)

    • This is a shortcut path that only works when the lakehouse is pinned (attached as default) to your notebook.
    • Example: Files/customers.parquet
    • Spark automatically knows where to look because the lakehouse is already attached.
    • Works with Spark only.

         3. File API Path (Local Mount Path)

    • When you pin a lakehouse to a notebook, Fabric automatically mounts it at /lakehouse/default/. This path points to that mount location.
    • Example: /lakehouse/default/Files/customers.parquet
    • It behaves like a local file system path, so it is mainly used with Pandas, because Pandas cannot understand ABFS paths directly. It needs a local-style path to read files.
    • Works with Pandas and any local file API (open(), os.listdir(), etc.)
    • Only available inside notebooks, not in Spark Job Definitions.

    Quick Summary:

    • ABFS Path = Full address, works from anywhere, no need to pin lakehouse.
    • Relative Path = Short path, works only in Spark, lakehouse must be pinned.
    • File API Path = Local mount path, works with Pandas, lakehouse must be pinned.

    Hope this helps!