trigger a pipeline on change

[arkiboys2]

I am ingesting data from sql server using fabric Data Factory

How is it possible to trigger a pipeline as soon as the sql server table has upsert?

Thank you

[NandanHegde]

Depending on SQL server version, you can use polybase feature and generate file and then trigger the pipeline via event

https://datasharkx.wordpress.com/2023/05/04/trigger-synapse-azure-data-factory-pipeline-via-sql-serv...

[mariussve1]

Hi,

I have a log table in data Warehouse that logs when sql table is updated. Then you can create a view that sets a flag to 1 or 0 depends if its updated today.

In the pipeline you can just create a lookup and if it returns 1 it continue else stop. Then you can schedule this every 15,30,60 min or What you think is best.

br

Marius

[NandanHegde]

Can you state which SQL server version you are using? That would help you suggest you a better way

[arkiboys2]

using azure sql server

[NandanHegde]

FOr Azure SQL database, please refer the below blog :

https://datasharkx.wordpress.com/2022/12/02/event-trigger-azure-data-factory-synapse-pipeline-via-az...

You can leverage the Fabric data pipeline rest api to trigger it or you can have a blank file created to event trigger the fabric data pipeline

[arkiboys2]

it look slike azure function is required?

even in fabric data factory there is no straight forward mechanism to trigger a DF pipeline?

thanks

[NandanHegde · Accepted solution]

You can use Azure function to generate the token at run time but you can even add the already generated token within database scoped credentails in Azure SQL database which would eliminate the need for Az function

you can go throgh this doc :

https://devblogs.microsoft.com/azure-sql/azure-sql-database-external-rest-endpoints-integration-publ...