Forum Discussion

AndrewPage's avatar
AndrewPage
New Member
3 months ago
Solved

DataFlow Gen 2 with KQL Engine Query URI as the destination

I'm trying to implement the Content Delivery Lifecycle using workspace Git integration- see https://learn.microsoft.com/en-us/fabric/cicd/git-integration/intro-to-git-integration   I need the desti...
  • svelde's avatar
    svelde
    3 months ago

    Hello AndrewPage ,

    what you experience with the connection is default behavior. The connection to an Eventhouse is static.

    If you want to distribute difference messages over multiple tables, there are two strategies:

    1. Create multiple connections and use staging and branching

     

     

    I have no experience with this solution and had to google a lot to get this explained.

     

    This could be the solution if you need to split the data towards multiple Eventhouses.

     

    2. use one destionation to a 'source' table in Eventhouse and use table update policies to split the data towards target tables

     

    This is a common scenario in Eventhouse where a medaillion architcture is used to split raw data over multiple silver tables.

     

    The raw/source/bronse table then gets a retention time so raw table rows are automatically deleted over time (otherwise you have duplicate data in both bronse and silver tables). 

     

    This works well within one eventhouse.

     

    --

    If this answer helps you, a thumbs-up or marking it as the accepted answer is appreciated. All community members with similar questions will benefit by doing so. Your contribution is highly appreciated.