Not able to send job_params using Apache Airflow FabricRunItemOperator() using job_params option

[KrishnaiahDarla]

I am using operator as follows:

FabricRunItemOperator(

                    task_id = job_name,

                    fabric_conn_id="ms_fabric_conn_id",

                    workspace_id=job_parameters['ms_fabric_workspace_id'],

                    item_id=job_parameters['ms_fabric_item_item_id'],

                    job_type=job_parameters['ms_fabric_item_job_type'],

                    #job_params= dict(name = "John", age = 36, country = "Norway"),

                    job_params={

                            'param1': 'value1'

                    },

                    wait_for_termination=True,

                    deferrable=True)

Apache Airflow is throwing below error:

Traceback (most recent call last):
  File "/home/kpiadmin/airflow_env/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 767, in _execute_task
    result = _execute_callable(context=context, **execute_callable_kwargs)
  File "/home/kpiadmin/airflow_env/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 733, in _execute_callable
    return ExecutionCallableRunner(
  File "/home/kpiadmin/airflow_env/lib/python3.10/site-packages/airflow/utils/operator_helpers.py", line 252, in run
    return self.func(*args, **kwargs)
  File "/home/kpiadmin/airflow_env/lib/python3.10/site-packages/airflow/models/baseoperator.py", line 422, in wrapper
    return func(self, *args, **kwargs)
  File "/home/kpiadmin/airflow_env/lib/python3.10/site-packages/apache_airflow_microsoft_fabric_plugin/operators/fabric.py", line 171, in execute
    raise FabricRunItemException(
apache_airflow_microsoft_fabric_plugin.hooks.fabric.FabricRunItemException: Item run 64246612-61c1-483d-9941-61cca9bc0378 has failed with status Failed.

Please help me how to pass parameters from this apache airflow FabricRunItemOperator() and access these parame

[DP700_Pro · Accepted solution]

Actualiza el plugin de Airflow. Es posible que estes utilizando una versión desactualizada del plugin "apache-airflow-microsoft-fabric-plugin. Considera actualizarlo a la versión más reciente para garantizar la compatibilidad y el soporte adecuados.

[sgol]

hi,

can you share the versions that solved this problem.

I am using Airflow version , 2.6.3 and "apache-airflow-microsoft-fabric-plugin==1.0.3"

As soon I try to pass job_params the DAG fails.

I see "run notebook payload is not well formatted." error. It looks like payload sent from Airflow to Notebook is not in the right way.

Here is how my DAG looks like, formatting is perfectly alright. If I take off job_params it works just fine.

with DAG(
        'landing_workflows.py',
        default_args=default_args,
        description='landing workflows',
        schedule="@hourly",
        catchup=False
) as dag:
    run_disk_storage = FabricRunItemOperator(
        task_id="storage_task",
        fabric_conn_id="fabric_conn",
        workspace_id="",
        item_id="",
        job_type="RunNotebook",
        job_params={
            'param1': 'value1'
        },
        wait_for_termination=True,
    )

    run_disk_storage​

[sgol]

Never mind guys.. The params dict needs to be structured like this ... if anyoine have a similar issue.

Hope this helps.

{
    "paramname": {
        "value": "paramvalue",
        "type": "string"
    }
}

[Chris_wright]

this worked for me thanks 

i can now orchestrate complex DAG with airflow in fabric

        job_params={

            "workspace": {

                "value": workspace,

                "type": "string",

            },

            "bronzelakehouse": {

                "value": bronzelakehouse,

                "type": "string",

            },

            "silverlakehouse": {

                "value": silverlakehouse,

                "type": "string",

            },

        }

[KrishnaiahDarla]

Thank you for solving my issue.

[v-pagayam-msft · Accepted solution]

Hi @KrishnaiahDarla ,
Thank you @DP700_Pro for the helpful response.
As DP700_Pro suggeted, consider updating it to the latest version to ensure compatibility and proper support. If our answer resolved your query, please mark it as "accept as solution" ,it will be helpful for other members of the community who have similar problems as yours to solve it faster.
If you need any further assistance, feel free to reach out.

Thank you for being a valued member in Microsoft Fabric Community Forum!

[DP700_Pro · Accepted solution]

Actualiza el plugin de Airflow. Es posible que estes utilizando una versión desactualizada del plugin "apache-airflow-microsoft-fabric-plugin. Considera actualizarlo a la versión más reciente para garantizar la compatibilidad y el soporte adecuados.