Forum Discussion
DataFlow Gen 2 with KQL Engine Query URI as the destination
Hello AndrewPage ,
what you experience with the connection is default behavior. The connection to an Eventhouse is static.
If you want to distribute difference messages over multiple tables, there are two strategies:
1. Create multiple connections and use staging and branching
I have no experience with this solution and had to google a lot to get this explained.
This could be the solution if you need to split the data towards multiple Eventhouses.
2. use one destionation to a 'source' table in Eventhouse and use table update policies to split the data towards target tables
This is a common scenario in Eventhouse where a medaillion architcture is used to split raw data over multiple silver tables.
The raw/source/bronse table then gets a retention time so raw table rows are automatically deleted over time (otherwise you have duplicate data in both bronse and silver tables).
This works well within one eventhouse.
--
If this answer helps you, a thumbs-up or marking it as the accepted answer is appreciated. All community members with similar questions will benefit by doing so. Your contribution is highly appreciated.
Super UserHello AndrewPage
welcom to this community forum.
The Dataflow Gen2 can forward messages to Eventhouse tables.
There, you can use table update policies to pick up new rows and transform/distribute/enzich them internally to other tables.
Or you can add Eventhouse CDC to pick up new rows in a Fabric Eventstream.
Finally, check out the Activator integration on the KQL queryset if you want to act on the incoming data.
You can change the retention time of the first table (eg. 30 days) so rows do not stay there for ever if not needed.
--
If this answer helps you, a thumbs-up or marking it as the accepted answer is appreciated. All community members with similar questions will benefit by doing so. Your contribution is highly appreciated.