Forum Discussion
notebookutils.fs.cp() cannot copy from Lakehouse shortcut to Files in Python notebook
- 1 month ago
Hi svencken,
notebookutils.fs.cp() is documented to support copying across file systems. In Fabric Python notebooks, cp() uses the same AzCopy-based mechanism as fastcp().
Microsoft also documents SharePoint folders as supported OneLake shortcut sources. However, the linked documentation does not explicitly list this specific SharePoint shortcut → Lakehouse Files AzCopyFailedException as a known limitation.
Since the two-step copy from the shortcut to notebook-local storage and then to Lakehouse Files succeeds, this approach can be used as a workaround in your environment.
NotebookUtils file system utilities for Fabric - Microsoft Fabric | Microsoft Learn
Create a OneDrive or SharePoint shortcut - Microsoft Fabric | Microsoft Learn
Unify Data Sources With OneLake Shortcuts - Microsoft Fabric | Microsoft LearnUnify Data Sources With OneLake Shortcuts - Microsoft Fabric | Microsoft Learn
Thank you.
Hi svencken,
I think there is one useful detail to add to the workaround already provided.
In Fabric Python notebooks, Microsoft's current NotebookUtils file-system documentation states that notebookutils.fs.cp() internally uses the same AzCopy-based mechanism as fastcp().
So in a Python 3.12 notebook, switching between cp() and fastcp() is not really testing two separate copy implementations. That also fits with both methods producing the same AzCopyFailedException in your case.
What I find interesting about your reproduction is:
- SharePoint shortcut -> notebook-local storage works
- notebook-local storage -> Lakehouse Files works
- SharePoint shortcut -> Lakehouse Files directly fails
That suggests the shortcut can be read and the Lakehouse destination can be written independently. The failure appears to be specific to the direct AzCopy-backed source/destination path.
Microsoft does document some specific cp() / fastcp() limitations for OneLake shortcuts, particularly S3/GCS shortcuts, but I do not see a current documented limitation stating that a SharePoint shortcut cannot be copied directly into Lakehouse Files.
SharePoint/OneDrive shortcuts themselves are now GA and are documented as accessible from Fabric workloads.
Before treating the two-step local copy as the permanent solution, I would test a minimal case and capture:
- the exact shortcut source path passed to cp()
- the exact Lakehouse Files destination path
- the SharePoint shortcut authentication type
- whether notebookutils.fs.exists() and notebookutils.fs.ls() succeed against that same source path
If those checks work and a single small file still fails only on the direct shortcut -> Files copy, I would raise it with Microsoft Support as a reproducible NotebookUtils/AzCopy interoperability issue.
For now, your shortcut -> local -> Lakehouse Files sequence is a reasonable workaround, but I would not consider the direct SharePoint-shortcut failure a documented product limitation based on the current Learn documentation.
AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.