Forum Discussion

KrishnaiahDarla's avatar
KrishnaiahDarla
New Member
1 year ago
Solved

Not able to send job_params using Apache Airflow FabricRunItemOperator() using job_params option

I am using operator as follows:     FabricRunItemOperator( task_id = job_name, fabric_conn_id="ms_fabric_conn_id", workspace_id=job_...
workspaces
  • DP700_Pro's avatar
    DP700_Pro
    1 year ago

    Actualiza el plugin de Airflow. Es posible que estes utilizando una versión desactualizada del plugin "apache-airflow-microsoft-fabric-plugin. Considera actualizarlo a la versión más reciente para garantizar la compatibilidad y el soporte adecuados.

  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi KrishnaiahDarla ,
    Thank you DP700_Pro for the helpful response.
    As DP700_Pro suggeted, consider updating it to the latest version to ensure compatibility and proper support. If our answer resolved your query, please mark it as "accept as solution" ,it will be helpful for other members of the community who have similar problems as yours to solve it faster.
    If you need any further assistance, feel free to reach out.


    Thank you for being a valued member in Microsoft Fabric Community Forum!

sgol's avatar
sgol
Icon for Advocate I rankAdvocate I
1 year ago
hi,
can you share the versions that solved this problem.
I am using Airflow version , 2.6.3 and "apache-airflow-microsoft-fabric-plugin==1.0.3"
As soon I try to pass job_params the DAG fails.
I see "run notebook payload is not well formatted." error. It looks like payload sent from Airflow to Notebook is not in the right way.
Here is how my DAG looks like, formatting is perfectly alright. If I take off job_params it works just fine.
 

 

 

with DAG(
        'landing_workflows.py',
        default_args=default_args,
        description='landing workflows',
        schedule="@hourly",
        catchup=False
) as dag:
    run_disk_storage = FabricRunItemOperator(
        task_id="storage_task",
        fabric_conn_id="fabric_conn",
        workspace_id="",
        item_id="",
        job_type="RunNotebook",
        job_params={
            'param1': 'value1'
        },
        wait_for_termination=True,
    )

    run_disk_storage​

 

 

sgol's avatar
sgol
Icon for Advocate I rankAdvocate I
1 year ago

 

Never mind guys.. The params dict needs to be structured like this ... if anyoine have a similar issue.

Hope this helps.

 

{
    "paramname": {
        "value": "paramvalue",
        "type": "string"
    }
}

 

  • Chris_wright's avatar
    Chris_wright
    Regular Visitor
    1 year ago

    this worked for me thanks 

    i can now orchestrate complex DAG with airflow in fabric

     

            job_params={
                "workspace": {
                    "value": workspace,
                    "type": "string",
                },
                "bronzelakehouse": {
                    "value": bronzelakehouse,
                    "type": "string",
                },
                "silverlakehouse": {
                    "value": silverlakehouse,
                    "type": "string",
                },
            }