Airflow dbt example fails on ODBC driver connect
I'm trying out the example https://learn.microsoft.com/en-us/fabric/data-factory/apache-airflow-jobs-dbt-fabric but I'm stuck at the airflow dbt job failing on ('08S01', '[08S01] [Microsoft][ODBC Driver 18 for SQL Server]TCP Provider: Error code 0x2746 (10054) (SQLDriverConnect)') which according to e.g. https://stackoverflow.com/questions/74708033/error-code-0x2746-10054-when-trying-to-connect-to-sql-server-2014-via-odbc-fro seems to be related to certificate trust. Anyone knows how to proceed? [2024-11-26, 10:23:26 UTC] {subprocess.py:90} INFO - [0m10:23:26 Finished running in 0 hours 0 minutes and 3.69 seconds (3.69s). [2024-11-26, 10:23:26 UTC] {subprocess.py:90} INFO - [0m10:23:26 Encountered an error: [2024-11-26, 10:23:26 UTC] {subprocess.py:90} INFO - Database Error [2024-11-26, 10:23:26 UTC] {subprocess.py:90} INFO - ('08S01', '[08S01] [Microsoft][ODBC Driver 18 for SQL Server]TCP Provider: Error code 0x2746 (10054) (SQLDriverConnect)') [2024-11-26, 10:23:26 UTC] {subprocess.py:94} INFO - Command exited with return code 2 [2024-11-26, 10:23:26 UTC] {taskinstance.py:1824} ERROR - Task failed with exception Traceback (most recent call last): File "/home/airflow/.local/lib/python3.8/site-packages/cosmos/operators/local.py", line 277, in execute result = self.build_and_run_cmd(context=context) File "/home/airflow/.local/lib/python3.8/site-packages/cosmos/operators/local.py", line 189, in build_and_run_cmd return self.run_command(cmd=dbt_cmd, env=env, context=context) File "/home/airflow/.local/lib/python3.8/site-packages/cosmos/operators/local.py", line 180, in run_command self.exception_handling(result) File "/home/airflow/.local/lib/python3.8/site-packages/cosmos/operators/local.py", line 76, in exception_handling raise AirflowException( airflow.exceptions.AirflowException: ('dbt command failed. The command returned a non-zero exit code 2. Details: ', '\x1b[0m10:23:22 Running with dbt=1.5.11', '\x1b[0m10:23:22 Registered adapter: fabric=1.5.0', '\x1b[0m10:23:22 Found 1 model, 0 tests, 0 snapshots, 0 analyses, 347 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics, 0 groups', '\x1b[0m10:23:22', '\x1b[0m10:23:26', '\x1b[0m10:23:26 Finished running in 0 hours 0 minutes and 3.69 seconds (3.69s).', '\x1b[0m10:23:26 Encountered an error:', 'Database Error', " ('08S01', '[08S01] [Microsoft][ODBC Driver 18 for SQL Server]TCP Provider: Error code 0x2746 (10054) (SQLDriverConnect)')") [2024-11-26, 10:23:26 UTC] {taskinstance.py:1345} INFO - Marking task as FAILED. dag_id=dbt_fabric_dag, task_id=nyc_trip_count.nyc_trip_count_run, execution_date=20241125T000000, start_date=20241126T102313, end_date=20241126T102326 [2024-11-26, 10:23:26 UTC] {standard_task_runner.py:104} ERROR - Failed to execute job 21 for task nyc_trip_count.nyc_trip_count_run (('dbt command failed. The command returned a non-zero exit code 2. Details: ', '\x1b[0m10:23:22 Running with dbt=1.5.11', '\x1b[0m10:23:22 Registered adapter: fabric=1.5.0', '\x1b[0m10:23:22 Found 1 model, 0 tests, 0 snapshots, 0 analyses, 347 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics, 0 groups', '\x1b[0m10:23:22', '\x1b[0m10:23:26', '\x1b[0m10:23:26 Finished running in 0 hours 0 minutes and 3.69 seconds (3.69s).', '\x1b[0m10:23:26 Encountered an error:', 'Database Error', " ('08S01', '[08S01] [Microsoft][ODBC Driver 18 for SQL Server]TCP Provider: Error code 0x2746 (10054) (SQLDriverConnect)')"); 69) [2024-11-26, 10:23:26 UTC] {local_task_job_runner.py:225} INFO - Task exited with return code 1 [2024-11-26, 10:23:26 UTC] {taskinstance.py:2653} INFO - 0 downstream tasks scheduled from follow-on schedule checkDAGRun via API
Hi, I am trying to find out how to run Airflow DAG via API call. When I called an API from swagger UI page, it works properly. (using /connection api as example for simplicity) However, if I called the same API from shell script or postman, it will always redirect me to microsoft login page. Is there a way to authenticate using shell script or do I need to change airflow Configuration Overrides to allow basic authentication (if possible)? Is it currently possible to run a DAG from a REST API call using basic authen or is there a way to authenticate using shell script ?Fabric Data
Microsoft Service Fabric is a distributed systems platform that allows developers to build, deploy, and manage scalable and reliable applications. Here's how it works with SQL: *Service Fabric Architecture:* Service Fabric is designed as a microservices architecture, where applications are broken down into smaller, independent services that communicate with each other. *SQL Integration:* Service Fabric provides several ways to integrate with SQL: 1. *Reliable Collections:* Service Fabric offers a built-in key-value store called Reliable Collections, which allows developers to store and retrieve data in a reliable and scalable manner. Reliable Collections can be used as a caching layer for SQL data. 2. *SQL Service Fabric Provider:* This provider allows developers to use SQL databases as a state provider for Service Fabric services. This enables developers to store and retrieve data from SQL databases using Service Fabric's programming model. 3. *Azure SQL Database:* Service Fabric can be integrated with Azure SQL Database, which provides a fully managed relational database service. Developers can use Azure SQL Database as a backend database for their Service Fabric Application
