Forum Discussion
notebookutils.fs.cp() cannot copy from Lakehouse shortcut to Files in Python notebook
I've been playing around with notebookutils.fs.cp() and notebookutils.fs.fastcp() in Fabric Python 3.12 notebooks.
This function seems to be unable to copy a file (e.g. PDF) from a SharePoint folder behind a Lakehouse shortcut to a directory in the Lakehouse Files. This produces a AzCopyFailedException.
Somewhat strangely, it is possible to use fs.cp() and fs.fastcp() to copy files from the shortcut to local file storage in the notebook environment and to then copy from this local file storage to the Lakehouse Files.
Has anyone come across a similar issue?
Hi svencken,
notebookutils.fs.cp() is documented to support copying across file systems. In Fabric Python notebooks, cp() uses the same AzCopy-based mechanism as fastcp().
Microsoft also documents SharePoint folders as supported OneLake shortcut sources. However, the linked documentation does not explicitly list this specific SharePoint shortcut → Lakehouse Files AzCopyFailedException as a known limitation.
Since the two-step copy from the shortcut to notebook-local storage and then to Lakehouse Files succeeds, this approach can be used as a workaround in your environment.
NotebookUtils file system utilities for Fabric - Microsoft Fabric | Microsoft Learn
Create a OneDrive or SharePoint shortcut - Microsoft Fabric | Microsoft Learn
Unify Data Sources With OneLake Shortcuts - Microsoft Fabric | Microsoft LearnUnify Data Sources With OneLake Shortcuts - Microsoft Fabric | Microsoft Learn
Thank you.
4 Replies
v-saisrao-msftCommunity Support
Hi svencken,
notebookutils.fs.cp() is documented to support copying across file systems. In Fabric Python notebooks, cp() uses the same AzCopy-based mechanism as fastcp().
Microsoft also documents SharePoint folders as supported OneLake shortcut sources. However, the linked documentation does not explicitly list this specific SharePoint shortcut → Lakehouse Files AzCopyFailedException as a known limitation.
Since the two-step copy from the shortcut to notebook-local storage and then to Lakehouse Files succeeds, this approach can be used as a workaround in your environment.
NotebookUtils file system utilities for Fabric - Microsoft Fabric | Microsoft Learn
Create a OneDrive or SharePoint shortcut - Microsoft Fabric | Microsoft Learn
Unify Data Sources With OneLake Shortcuts - Microsoft Fabric | Microsoft LearnUnify Data Sources With OneLake Shortcuts - Microsoft Fabric | Microsoft Learn
Thank you.
- v-saisrao-msft
HI svencken,
Have you had a chance to review the solution we shared earlier? If the issue persists, feel free to reply so we can help further.
Thank you.
- v-saisrao-msft
HI svencken,
Checking in to see if your issue has been resolved. let us know if you still need any assistance.
Thank you.
- ShivekMaharaj
Community Champion
Hi svencken,
I think there is one useful detail to add to the workaround already provided.
In Fabric Python notebooks, Microsoft's current NotebookUtils file-system documentation states that notebookutils.fs.cp() internally uses the same AzCopy-based mechanism as fastcp().
So in a Python 3.12 notebook, switching between cp() and fastcp() is not really testing two separate copy implementations. That also fits with both methods producing the same AzCopyFailedException in your case.
What I find interesting about your reproduction is:
- SharePoint shortcut -> notebook-local storage works
- notebook-local storage -> Lakehouse Files works
- SharePoint shortcut -> Lakehouse Files directly fails
That suggests the shortcut can be read and the Lakehouse destination can be written independently. The failure appears to be specific to the direct AzCopy-backed source/destination path.Microsoft does document some specific cp() / fastcp() limitations for OneLake shortcuts, particularly S3/GCS shortcuts, but I do not see a current documented limitation stating that a SharePoint shortcut cannot be copied directly into Lakehouse Files.
SharePoint/OneDrive shortcuts themselves are now GA and are documented as accessible from Fabric workloads.
Before treating the two-step local copy as the permanent solution, I would test a minimal case and capture:
- the exact shortcut source path passed to cp()
- the exact Lakehouse Files destination path
- the SharePoint shortcut authentication type
- whether notebookutils.fs.exists() and notebookutils.fs.ls() succeed against that same source path
If those checks work and a single small file still fails only on the direct shortcut -> Files copy, I would raise it with Microsoft Support as a reproducible NotebookUtils/AzCopy interoperability issue.For now, your shortcut -> local -> Lakehouse Files sequence is a reasonable workaround, but I would not consider the direct SharePoint-shortcut failure a documented product limitation based on the current Learn documentation.
AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.
