Lookup task on Datawarehouse execution issue since 1 Jan 2025

[Testaaaaaa]

Hello,

Our Lookup activity on a table in Datawarehouse is having issue to execute simple SQL query since the 1st of January.

The look up stays in executing state until either timeout or someone needs to manually query the datawarehouse - which is not sustainable.

We are currently pausing capacity at night time and waking it up every morning to run morning ETL process. This look up is one of the first step to be executed.

Is anybody else experiencing this kind of issue? 

[Testaaaaaa]

Hello,

I have used a work around by a combination of running a SQL statement to "warm up" the DW and by delaying by an extra 10 min the start of the ETL process.

Since then I have not had any issue.

@v-ssriganesh Thank you for your help

[Testaaaaaa]

Hi @v-ssriganesh 

Thank you for replying, are you able to tell me how to access the Operations Manager event log on MS Fabric Datawarehouse?

• I have just turned on MS Fabric and 5 minute later, the morning ETL stats, so ressources is not a problem as Lookup is the first step of the process. This ETL used to work with no issue for about 3 months without issue
• Query is very simple - returning up to1 row, 5 columns as a select statement : 

SELECT * FROM [dbo].[Pipeline_Run_Status] WHERE is_running = 'Y'

• I have started manually the pipeline when getting to work this morning and lookup worked with no issue. It seems a delay in MS Fabric DW being ready upon starting MS Fabric capacity? 

[v-ssriganesh · Accepted solution]

Hi @Testaaaaaa,

Thank you for your follow-up. To access the Operations Manager event log on MS Fabric Datawarehouse, you can follow this step:

• On your Datawarehouse server, open the Event Viewer application.In the Event Viewer, expand the "Applications and Services Logs" section, then look for "Operations Manager". This log will contain errors and events related to the Datawarehouse.

Regarding the delay you're experiencing, it seems like the MS Fabric Datawarehouse might need some additional time to fully initialize after starting the capacity. Here are a few suggestions to address this:

• Try increasing the time between starting the MS Fabric capacity and the beginning of your ETL process. This will give the Datawarehouse more time to be fully ready.
• Ensure that the resources allocated to the Datawarehouse are sufficient and that there are no bottlenecks during the initialization phase.
• Verify if there are any updates or patches available for MS Fabric that might address this initialization delay.

Since your query is simple and works fine when manually started, it indicates that the issue is likely related to the readiness of the Datawarehouse rather than the query itself.

If you find this information helpful, kindly accept it as a solution and leave a "Kudos" to aid other members in locating it more easily.
Thank you.

[v-ssriganesh]

Hi @Testaaaaaa,

Thank you for contacting the Microsoft Fabric community forum.

you're experiencing an issue with the Lookup activity in your Datawarehouse, where it stays in the executing state until it times out or requires manual intervention.

Here are some steps to troubleshoot the issue:

• Start with the Operations Manager event log on your Data Warehouse server. Look for errors related to the Data Warehouse, especially from ETL jobs. In the Service Manager console, check the status of your ETL jobs and the MP Sync Job. They should be either "Running" or "Not Started."
• Make sure your Data Warehouse has enough resources when it wakes up in the morning. Insufficient resources can lead to delays or timeouts. Verify there are no network issues or errors connecting to SQL Server. Ensure the instance name is correct and that remote connections are allowed.
• If your SQL query is complex, try breaking it down into smaller parts. Also, ensure your indexes are maintained and statistics are up to date.
• Check if there are any known problems with the Lookup activity in your environment. Sometimes, adding an alias to your SQL query can help.

If these steps do not resolve the issue, kindly provide more details about your environment and any error messages you are encountering.

If this helps then please Accept it as a solution and dropping a "Kudos" so other members can find it more easily.
Hope this works for you!
Thanks.