Create an activity in data pipeline to do metadata refresh of sql analytics endpoint

[Imjkrao]

The expectation is to add the "metadata refresh" as a configurable activity in the data pipeline. So, when we do a data load operation, we can run this metadata refresh activity before refreshing the semantic model, which would guarantee that the updated data is in the report.

Issue: Whenever I insert new data into the Lakehouse table, on random occasions, the SQL analytics endpoint seems to go out of sync. Due to that, the semantic model won't get updated and the report still shows the old data.

In the data pipeline, I have added a 120-second wait activity after the data is loaded and before I refresh the semantic model, but this seems to be a hit-and-miss as well.

However, if I click on the metadata refresh button on the lakehouse and then refresh the semantic model, this works perfectly well. So, turn that button in the SQL analytics endpoint into an activity in the data pipeline.

Thanks