Forum Discussion

AndyAtAsos's avatar
AndyAtAsos
New Member
2 years ago

Cosmos CDC - updates not flowing

Hi,

I have the following topology, where I am trying to stream orders that are written to a Cosmos DB, to a Lakehouse and External Connector.

Q1: What is the cause of the Warning on the Cosmos DB source?
Q2: Is the warning detail written anywhere (e.g. Fabric UI, Log Analytics)?

Q3: When I connect a destination, the initial documents flow through to the custom endpoint. But if I write further documents, nothing appears. Why is this?
Q4: When nothing occurs for 2 hours, the eventstream idles. What causes it to activate again? Writing documents appears not to activate the eventstream.

Thank you

Eventstream

9 Replies

  • AndyAtAsos's avatar
    AndyAtAsos
    New Member
    2 years ago

    Hi,

    I have the following topology:

    I cannot get any events to flow to the Lakehouse and get written to a table. The table isn't created either. If I create a table myself, events do not get written.


    I can see events flowing to the custom endpoint.

    1. Any ideas what the problem is?

    2. Is there a way to debug this?

     

    Thank you

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      Hi AndyAtAsos ,

      The reason you might not see any data events to flow to the Lakehouse is that your data existed before the Lakehouse was added. You can try adding a new document to your Cosmos DB. This should trigger a new CDC event, which should then flow into your Lakehouse. Once data starts flowing to the Lakehouse, you should see the table created.

      • Sudip's avatar
        Sudip
        Frequent Visitor
        2 years ago

        is this issue resolved ?? I am having a similar issue. 

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi AndyAtAsos ,

    Please click on the Warning tile and check if any error or warning notices appear at the bottom of the screen?

    Also, it is the first tile, the data source tile, that appears with the warning prompt. Please make sure that your data source is available and the connection is configured correctly. If your data source is incorrect, even if you configure it correctly later, it will not stream any data to your final destination.

    Best Regards,
    Dino Tao
    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

  • AndyAtAsos's avatar
    AndyAtAsos
    New Member
    2 years ago

    Thank you for replying.

    There is no further detail displayed alongside the warning icon on the source Cosmos.

    If you hover over the tile and the status field in the Details tab, it doesn't display anything.

    I believe the source is connected and working because if you view data preview on the eventstream it shows data and the last updated date is incremented.

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      Hi AndyAtAsos ,

      Please check what you can see in the logs:

      The situation you are facing is a warning and not an Error, so it may indeed not show Error details, then we need to see if we can get some information by reading the contents of the logs.

      Best Regards,
      Dino Tao

      • AndyAtAsos's avatar
        AndyAtAsos
        New Member
        2 years ago

        Hi,

        Unfortunately there is nothing in the logs.

        Is anything logged elsewhere? e.g. Log Analytics?

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi @AndyAtAsos ，

    May we kindly ask you to provide your datasource ID? We would like to look into the issue you're facing.

     

    Additionally, you might want to try adjusting the time range of the runtime logs to see if there are any logs from your Cosmos DB.

     

     

    Thanks,

    Wenyang