Forum Discussion

Nur's avatar
Nur
Frequent Visitor
1 year ago
Solved

InvalidTemplate error

Hi Fabric aficionado,

 

I am pretty new in Microsoft Fabric. I encountered this error when running my pipeline (at Filter activity stage):

Error code
InvalidTemplate
Failure type
User configuration issue
Details
The execution of template action 'FilterNon-Empty Partition Column' failed: The evaluation of 'query' action 'where' expression '@not(empty(item().partition_column)) ' failed: 'The expression 'not(empty(item().partition_column)) ' cannot be evaluated because property 'partition_column' doesn't exist, available properties are 'source_schema_name, source_table_name, bronze_file_directory, bronze_file_path, bronze_table_path, table_name, lakehouse_name, select_statement'.

The Filter is connected from a Lookup activity. So, how can I include partition_column property together with the available properties listed above?

If you need more information, please let me know. Thank you in advance!

CICD
  • v-veshwara-msft's avatar
    v-veshwara-msft
    1 year ago

    Hi Nur ,

    Thank you for reaching out to the Microsoft Fabric community.

    The error you're encountering typically happens when either:

    The partition_column doesn’t exist in the underlying source dataset, or

    The partition_column is present in the source, but the Lookup query isn’t selecting it.

    To resolve this, you can follow the steps below:

    >>Ensure partition_column Is Returned by Lookup.
    If you are using Query in Lookup activity update your query to explicitly include partition_column.

    For example:

    SELECT source_schema_name, table_name, partition_column
FROM [test_table]


    If partition_column doesn’t exist in the source table schema, you can still include it as a placeholder:

    SELECT source_schema_name, table_name, '' AS partition_column
FROM [test_table]


    >>Set “First row only” to False
    In the Lookup activity settings, ensure “First row only” is set to false so that the output is an array (which the Filter activity requires).

     

    >>Configure the Filter Activity
    In the Filter activity:

    Items:

    @activity('YourLookupActivityName').output.value
    Condition:

    @not(empty(item().partition_column))
    This ensures that only records with a non-empty partition_column will be passed through.

     

    To assist you further, could you please confirm:
    • What is the source of your data used in the Lookup activity (e.g. Lakehouse, SQL Database, etc.)?
    • Are you using a custom query or selecting a table directly in the Lookup?
    • If partition_column actually exists in the source?
    These details will help us determine the best approach to include or work around the partition_column.

    Hope this helps. Please reach out for further assistance.
    If this post helps, then please consider to Accept it as the solution to help the other members find it more quickly and a kudos would be appreciated.

    Thanks,
    Vinay.

     

2 Replies

  • v-veshwara-msft's avatar
    v-veshwara-msft
    Community Support
    1 year ago

    Hi Nur ,

    Thank you for reaching out to the Microsoft Fabric community.

    The error you're encountering typically happens when either:

    The partition_column doesn’t exist in the underlying source dataset, or

    The partition_column is present in the source, but the Lookup query isn’t selecting it.

    To resolve this, you can follow the steps below:

    >>Ensure partition_column Is Returned by Lookup.
    If you are using Query in Lookup activity update your query to explicitly include partition_column.

    For example:

    SELECT source_schema_name, table_name, partition_column
FROM [test_table]


    If partition_column doesn’t exist in the source table schema, you can still include it as a placeholder:

    SELECT source_schema_name, table_name, '' AS partition_column
FROM [test_table]


    >>Set “First row only” to False
    In the Lookup activity settings, ensure “First row only” is set to false so that the output is an array (which the Filter activity requires).

     

    >>Configure the Filter Activity
    In the Filter activity:

    Items:

    @activity('YourLookupActivityName').output.value
    Condition:

    @not(empty(item().partition_column))
    This ensures that only records with a non-empty partition_column will be passed through.

     

    To assist you further, could you please confirm:
    • What is the source of your data used in the Lookup activity (e.g. Lakehouse, SQL Database, etc.)?
    • Are you using a custom query or selecting a table directly in the Lookup?
    • If partition_column actually exists in the source?
    These details will help us determine the best approach to include or work around the partition_column.

    Hope this helps. Please reach out for further assistance.
    If this post helps, then please consider to Accept it as the solution to help the other members find it more quickly and a kudos would be appreciated.

    Thanks,
    Vinay.

     

    • Nur's avatar
      Nur
      Frequent Visitor
      1 year ago

      Hi Vinay,

       

      Thank you for your answer!I have solved the problem yesterday 🙂

      It is indeed a silly problem where I should add partition_column in the notebook. huhuhu. Thank you again, have a nice day 🙂