Forum Discussion

KrishnaiahDarla's avatar
KrishnaiahDarla
New Member
1 year ago
Solved

Not able to send job_params using Apache Airflow FabricRunItemOperator() using job_params option

I am using operator as follows:     FabricRunItemOperator( task_id = job_name, fabric_conn_id="ms_fabric_conn_id", workspace_id=job_...
workspaces
sgol's avatar
sgol
Icon for Advocate I rankAdvocate I
1 year ago
hi,
can you share the versions that solved this problem.
I am using Airflow version , 2.6.3 and "apache-airflow-microsoft-fabric-plugin==1.0.3"
As soon I try to pass job_params the DAG fails.
I see "run notebook payload is not well formatted." error. It looks like payload sent from Airflow to Notebook is not in the right way.
Here is how my DAG looks like, formatting is perfectly alright. If I take off job_params it works just fine.
 

 

 

with DAG(
        'landing_workflows.py',
        default_args=default_args,
        description='landing workflows',
        schedule="@hourly",
        catchup=False
) as dag:
    run_disk_storage = FabricRunItemOperator(
        task_id="storage_task",
        fabric_conn_id="fabric_conn",
        workspace_id="",
        item_id="",
        job_type="RunNotebook",
        job_params={
            'param1': 'value1'
        },
        wait_for_termination=True,
    )

    run_disk_storage​

 

 

sgol's avatar
sgol
Icon for Advocate I rankAdvocate I
1 year ago

 

Never mind guys.. The params dict needs to be structured like this ... if anyoine have a similar issue.

Hope this helps.

 

{
    "paramname": {
        "value": "paramvalue",
        "type": "string"
    }
}

 

  • Chris_wright's avatar
    Chris_wright
    Regular Visitor
    1 year ago

    this worked for me thanks 

    i can now orchestrate complex DAG with airflow in fabric

     

            job_params={
                "workspace": {
                    "value": workspace,
                    "type": "string",
                },
                "bronzelakehouse": {
                    "value": bronzelakehouse,
                    "type": "string",
                },
                "silverlakehouse": {
                    "value": silverlakehouse,
                    "type": "string",
                },
            }