Power BI is turning 10! Tune in for a special live episode on July 24 with behind-the-scenes stories, product evolution highlights, and a sneak peek at what’s in store for the future.
Save the dateEnhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.
After successfully starting the Hello Airflow job example, I am trying a slightly modified version of this tutorial, in a new Airflow job:
https://learn.microsoft.com/en-us/fabric/data-factory/apache-airflow-jobs-dbt-fabric
The modification is that I am connecting to the default Warehouse with sample data, instead of the mentioned Contoso warehouse.
The problem is that the DAG does not run, after trying multiple schedule intervals.
Hopefully I have configured the DAG correctly, the only doubt I have is the database field in the profiles YAML file. Should that be the name of the warehouse?
EDIT:
I just discovered that there is a schedule interval set to daily, which means that the DAG will run at midnight on the day it is started, right? Is this a particulary beginner friendly setting for a tutorial? The tutorial also says to "press play", but this is a scheduled DAG right?
EDIT 2: I have set the schedule interval to continuous, and start date two days ago, catchup to False still no DAGs are run. Where can I find the logs?
Screenshots:
Solved! Go to Solution.
Hi @BiJoe,
Yes, you're correct! When a DAG is set with a daily schedule interval, it will run at midnight on the day it is started. While this can be a common setting, it might not be the most beginner-friendly option for a tutorial since it can lead to confusion about immediate execution.
If you still encounter issues with your DAGs not running, double-check your configurations and the Airflow scheduler status.
Let me know if you need further assistance!
Best regards,
Sai Prudhvi Neelakantam
Data Engineer | 3x Microsoft Certified
LinkedIn: in/saiprudhvineelakantam | geekindata.com
If this helps, kindly mark this as the accepted solution! 👍
Hi @BiJoe ,
Could you please tell me what was the exact solution? I have the same problem you had and I cannot really understand from the 'solved' reply what was the solution....
Hi @Floris_Loeffen I doubt that I set the status so "Solved" myself, not really sure. It was not working properly IIRC. Haven't touched this since back then.
Hi @BiJoe
In Fabric, the database field should be the name of the warehouse you want to connect to. Make sure it matches the default warehouse name in the Fabric with the sample data.
If the scheduling interval is set to daily, the DAG will run at midnight (UTC time) on the day it starts.
Even if the DAG is a scheduled task, you can trigger it manually by pressing the Play button in Airflow UI.
About Monitoring DAG:
Regards,
Nono Chen
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
@Anonymous thanks for replying.
Unfortunately there is no Play button in Airflow UI for the DAG I have replicated from the tutorial.
Thanks for the link about monitoring DAGs, but that does not work so long the DAG does not run properly.
Hi @BiJoe,
Yes, you're correct! When a DAG is set with a daily schedule interval, it will run at midnight on the day it is started. While this can be a common setting, it might not be the most beginner-friendly option for a tutorial since it can lead to confusion about immediate execution.
If you still encounter issues with your DAGs not running, double-check your configurations and the Airflow scheduler status.
Let me know if you need further assistance!
Best regards,
Sai Prudhvi Neelakantam
Data Engineer | 3x Microsoft Certified
LinkedIn: in/saiprudhvineelakantam | geekindata.com
If this helps, kindly mark this as the accepted solution! 👍