Uisng Fabric jupyter notebook to load/push data from/to Snowflake
Hello Farbic Community I'm new to using Fabric and am exploring its capabilities. I understand that we can load data from Snowflake into a Fabric Notebook via Data Lakehouse. I'm curious if it's possible to use a Python script in a Fabric notebook to directly load data from Snowflake into the notebook without involving a data lakehouse. In addition to direct data import from Snowflake, I'm also interested in whether data can be pushed back to Snowflake. If direct data import/export to and from Snowflake isn't feasible with a Python script in a Fabric notebook, is it possible to accomplish this using Dataflow or Data Pipeline? I would appreciate any suggestions and guidance.
Starter pool limitation alternative workarounds
Starter pool limitation: Workspaces with managed virtual networks (VNets) can't access starter pools. This category encompasses workspaces that use managed private endpoints or are associated with a Fabric tenant enabled with Azure Private Links and have executed Spark jobs. Such workspaces rely on on-demand clusters, taking three to five minutes to start a session from mentioned article https://learn.microsoft.com/en-us/fabric/security/security-managed-private-endpoints-overview#limitations-and-considerations Is there any workaround or alternative for this ?What is a recommended best practice around Spark workloads?
Deployment pipeline troubles
Hello I some questions about deployment pipeline. I using notebooks to integrate data in m'y lakehouse. When a deploy my deployment pipeline to my prod workspace, the lakehouse in the notebooks is still thé lakehouse of my development workspace. How can i make the change ? Then when i deploy my datawarehouse whitch is created with stocked procedures, i Can not open my pipeline whitch contain my stocked procedures from my production workspace. After Reading a lot i think that there IS a lot of things whitch are in preview but if you are some informations. It will be great. Thank for your help. Ben
Unable to create a shortcut in Fabric to my serverless Synapse with private endpoints ADLS Gen 2 DL
I have a serverless Synapse deployment with a bunch of parquet and CSV files in ADLS Gen 2 storage account for my synapse. The synapse is configured to use private endpoint connections. I have created a new PE and approved it for the storage account. When I try to create a shortcut in Fabric it fails no matter what connection method that I chose saying invalid credentials and I am unable to find any further detail/error messages to help me resolve this issue. Please help! I have also tried connecting via spark and get unauthorized access errors despite ensuring my user has the Storage Blob Data Contributor role. Spark error: Caused by: Operation failed: "This request is not authorized to perform this operation.", 403, HEAD, ... fyi- My synapse deployment config follows the published End to End Analytics: https://learn.microsoft.com/en-us/azure/architecture/example-scenario/dataplate2e/data-platform-end-to-end?tabs=portal
Add new and last modified records to a delta table in Lakehouse Fabric from parquet files
I have some parquet files related to GeneralLedger records in a Lakehouse and I want using pipeline to add only new and last modified records to a Delta table. What should I do? I used combination of lookup activity and copy data, but doesn't work for me. Any idea?
