Notebook fails to read newly created delta table

[jchappell99]

I have a pipeline that runs every morning.  The PL executes 5 dataflows that pull data into our bronze layer lakehouse.  After all 5 have succeeded, the pipeline calls a notebook that moves data from Bronze to Silver.  The first step in the notebook is reading the delta tables that were just created.  The notebook fails with the error: Notebook execution failed at Notebook service with http status code - '200', please check the Run logs on Notebook, additional details - 'Error name - AnalysisException, Error value - [TABLE_OR_VIEW_NOT_FOUND] The table or view `LH_MillSalesBronze`.`Elixir_tbl_invc` cannot be found. Verify the spelling and correctness of the schema and catalog.

When I see the error, I go run the notebook and there is no error.  The issue is it doesn't fail every morning.  Some days it works and some days it fails.  I even put a 2 minute pause into the pipeline and that doesn't help as it still fails some days.

Looking for help with this.  Due to another issue where the notebook was duplicating the number of rows after reading the delta table(another issue), I am dropping the delta tables and recreating daily.

Appreciate any help I can get.

[jchappell99]

Fails most days.  However, it has ran successfully 2 days in the last week. Error is below.  The table exists as it was created in the previous step of the pipeline(using dataflow gen 2).  I even put a "pause"  in the pipeline of 2 minutes and that didn't resolve the issue.  I appreciate your help.

Notebook execution failed at Notebook service with http status code - '200', please check the Run logs on Notebook, additional details - 'Error name - AnalysisException, Error value - [TABLE_OR_VIEW_NOT_FOUND] The table or view `LH_MillSalesBronze`.`Elixir_tbl_invc` cannot be found. Verify the spelling and correctness of the schema and catalog.
If you did not qualify the name with a schema, verify the current_schema() output, or qualify the name with the correct schema and catalog.
To tolerate the error on drop use DROP VIEW IF EXISTS or DROP TABLE IF EXISTS.;
'UnresolvedRelation [LH_MillSalesBronze, Elixir_tbl_invc], [], false
' :

[Anonymous]

Hi @jchappell99 ,

Apologize for the issue you are facing. The best course of action is to open a support ticket and have our support team take a closer look at it.

Please reach out to our support team so they can do a more thorough investigation on why this it is happening: Link 

After creating a Support ticket please provide the ticket number as it would help us to track for more information.

Hope this helps. Please let us know if you have any other queries.

[jchappell99]

Support request 2406170010003657

[Anonymous]

Hi @jchappell99 ,

Thanks for sharing the support ticket.

Please allow some time, so team can check and provide a resolution.

In case if you got a resolution please do share with the community as it can be helpful to others .

[Anonymous]

Hi @jchappell99 ,

Thanks for using Fabric Community. Apologies for the issue you are facing.
Can you please confirm whether you are observing this issue frequently or intermittently?