how to set an execution order for tables in Fabric ?

[Franc996]

Good afternoon everyone,

I have a problem with Fabric. I have a pipeline and I need to ingest the tables in a specific order. I have set an execution order column in a table on my database to ensure the correct order for ingestion. However, it does not consider the execution order and instead uses the parameterid (a unique and sequential index of the tables). I cannot use the parameterid because it does not have the correct order. I managed to make the lookup that retrieves the parameters for the pipeline follow the correct execution order, but the output returns it in the order of the parameter_id.

[v-jingzhan-msft]

Hi @Franc996 

Have you resolved this issue? If any of the answers provided were helpful, please consider accepting them as a solution. If you have found other solutions, we would greatly appreciate it if you could share them with us. Thank you!

Best Regards,
Jing

[FabianSchut]

Hi, can you share more details about your current setup. Do you have some screenshots and code that you use to get the order?

[Franc996]

 

Unfortunately, I can't share screenshots, but I'll try to be as clear as possible. The first element of my pipeline is a lookup that retrieves data from a view in my database. This view contains all the necessary parameters to pass to Fabric (which are taken from other tables within my database). Among these parameters, there is also a parameter ID, which is a unique index assigned to each table, and an execution order, which is unique for each table and represents the order I am trying to achieve. The view is sorted with an ORDER BY execution_order ASC, but the output is returned in the order of the parameter ID. This lookup is then connected to two filters that filter the tables based on other parameters before entering two for each loops based on the filter. The problem is in the lookup, which returns the output incorrectly. When I try to preview the lookup results, they are correctly ordered by execution_order.

[FabianSchut]

Is it possible that the order gets changed again in the filter step? Or does the actual lookup activity output already gives a wrong order?

[Franc996]

I confirm that the output returned by the lookup does not match the correct order I have in the view.

[FabianSchut]

Instead of selecting the view directly from the dropdown, can you select it using a query? You can then explicitely order by the same column as the one used in the view. Your query will look something like this in the lookup:
select * from VIEW_NAME order by ORDER_COLUMN
When I used a query to order by a column, that order what preserved in the output of the lookup (I tested with a table instead of a view, but I think it will work the same.