topic Re: Shortcut of mirrored database won't update in Data Engineering

Shortcut of mirrored database won't update

felicial — Fri, 04 Apr 2025 08:48:34 GMT

I created a Cosmos DB for NoSQL account and container and enabled mirroring.

Then I created a shortcut to run a Notebook on my data.

I add new data to Cosmos DB and thhe mirror upadtes. However, the shortcut won't. I need to recreate it everytime to get new data.

I have two questions:

1) Why do I have to create a shortcut instead of running the Notebook over the mirrored database?

2) Why won't it update?

Thanks for your support.

Re: Shortcut of mirrored database won't update

Anonymous — Fri, 04 Apr 2025 12:18:19 GMT

Hell0 felicial

Thank you for reaching out to the Microsoft Fabric Forum Community.

the shortcut doesn't dynamically update when new data is added or mirrored in your Cosmos DB. this might be due to shortcut is using cached data or The data refresh mechanism isn't properly configured to reflect updates in real-time

to avoid recreating shortcuts in your Cosmos DB integration with notebooks. check below resources to track changes to your data and let me know if this helps?
The Change Feed feature(https://learn.microsoft.com/en-us/azure/cosmos-db/change-feed) in Cosmos DB allows you to listen for changes in your Cosmos DB container and process new or updated data in real-time. This is useful when you want to track and handle changes automatically.
Using the Change Feed, you can continuously stream new data from Cosmos DB into your notebook environment without needing to manually recreate any shortcuts.

If this information is helpful, please “Accept it as a solution” and give a "kudos" to assist other community members in resolving similar issues more efficiently.
Thank you.

Re: Shortcut of mirrored database won't update

felicial — Fri, 04 Apr 2025 12:29:09 GMT

Hi Prasannag,

Thanks for your answer.

My intention is to use the mirroring feature in Fabric, which relies on the continous backup of Cosmos DB and not the change feed. I am trying to build the Notebook in Fabric.

What options are there for "data refresh mechanisms"? I thought it was automatic.

Thanks!

Re: Shortcut of mirrored database won't update

nilendraFabric — Fri, 04 Apr 2025 14:26:26 GMT

Hi felicial

You can access the mirrored database directly from notebooks using Spark SQL queries

df = spark.sql("SELECT * FROM YourMirroredDatabase.YourTable")
display(df)

This approach will give you latest data.with a typical latency of up to 60 seconds

Ideally both Shortcut and mirrored DB should provide the same infomation.

Re: Shortcut of mirrored database won't update

felicial — Fri, 04 Apr 2025 14:33:08 GMT

Thanks, I will try using the Notebook directly on the mirror. The shortcut won't update...

Will post the results. Thanks a lot!