Forum Discussion

JosueMolina's avatar
JosueMolina
Helper III
1 year ago
Solved

Notebook Activities with a custom environment stopped working when run from a Data Pipeline suddenly

I have a Pipeline that runs 9 notebooks, each notebook in a different workspace with it's own lakehouse and it's own nearly identical custom environment.    This pipeline had been running with no i...
data engineering
data pipeline
environment
notebook
  • v-prasare's avatar
    v-prasare
    1 year ago

    Hi JosueMolina, OZaharia, Tough 

    Sorry for the inconvenience caused here.

     

    As JosueMolina mentioned this was previously working, joanfatz please do raise a support ticket this will allow the a dedicated team to investigate and provide a resolution. Please include details of your setup, test results, and the impact on your workflows when submitting the ticket.

    How to create a Fabric and Power BI Support ticket - Power BI | Microsoft Learn

     

     

     

    Thanks,

    Prashanth Are

    MS Fabric Community Support

     

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query.

JosueMolina's avatar
JosueMolina
Helper III
1 year ago

I did some tests. The Pipeline is in Workspace A and each pair of Notebook and Environment is in a different Workspace; let's call one of these Workspace B. 

Workspace A has an F16 Capacity and Workspace B has an F64 Capacity. Doing a test of moving Workspace B to the F16 Capacity makes the error disappear.

 

Sadly, whatever was changed internally broke a previously working pipeline and moving the Workspace B to an F16 is not a solution for us.

Workspace B is a Gold layer workspace with a Lakehouse and we require an F64 to exist so users do not require a Pro license to consume our reports using this data.

Tough's avatar
Tough
New Member
1 year ago

Hey, just commenting to let you know you aren't alone. I support a Fabric environment running on an F64 capacity. About two weeks ago the pipeline completely stopped running and fails on the first step with the same error you are encountering. 

I've been trying to troubleshoot it myself, as the pipeline creator is currently on leave. I haven't made any progress and don't have another capacity to easily swap the pipeline to. Short term I have been running each step manually to keep the data fresh... But this is terrible. 

It was working perfectly fine and only minor changes have been made to the notebooks. 

Every notebook step and environment used in the pipeline is on F64 capacity out of Canada Central.

 

I haven't had dedicated time to call submit a support request, but looks like I'll be doing that tomorrow.

 

Thanks for posting this, was tough to find any info on this error.