Forum Discussion
Eventhouse data transformation
Hi, I'm new to Fabric and I was learning Eventhouse.
I just wanted to know if I ingest data into Eventhouse and I was to make some transformation on that data, how do I do it?
Should I create a shortcut of that eh kql db into a lakehouse, attach it on a notebook, make some transformations and then dump it into a ware house?
And in a scenario like this where data is continuously ingesting into the EH, should I be using spark structured streaming?
Please let me know the correct procedure and best one as market standard.
Hello svk666
If you're ingetsing the live data in to event house, you no need to create EH shortcut and no neeed to go for Lakehouse for tarsformation.
In evenhous eit self we have KQL, by using this you can transform the data, Once the data is transfomed you can build the dashboard from eventhouse itself. Pls read this thread https://community.fabric.microsoft.com/t5/Real-Time-Intelligence-Community/Eventhouse-and-its-best-practices/ba-p/5043560? it might helps you
Thank you!
Did I answer your question? Mark my post as a solution!
Proud to be a Super User!Hi svk666 ,
There are multiple ways in which you can do transformation either you can use event stream or directly ingesting to event house as per suparnababu8 . If there is any minimal transformation you can leverage predefined options in event stream,
1. You can leverage SQL based transformation if you are more inclined with sql
2.Ingest the data into Eventhouse and use notebook-based transformations directly within the Event Stream—there’s no need to create shortcuts or add separate logic.
Regards,
Srisakthi
5 Replies
Hi svk666 ,
There are multiple ways in which you can do transformation either you can use event stream or directly ingesting to event house as per suparnababu8 . If there is any minimal transformation you can leverage predefined options in event stream,
1. You can leverage SQL based transformation if you are more inclined with sql
2.Ingest the data into Eventhouse and use notebook-based transformations directly within the Event Stream—there’s no need to create shortcuts or add separate logic.
Regards,
Srisakthi
Hello svk666
If you're ingetsing the live data in to event house, you no need to create EH shortcut and no neeed to go for Lakehouse for tarsformation.
In evenhous eit self we have KQL, by using this you can transform the data, Once the data is transfomed you can build the dashboard from eventhouse itself. Pls read this thread https://community.fabric.microsoft.com/t5/Real-Time-Intelligence-Community/Eventhouse-and-its-best-practices/ba-p/5043560? it might helps you
Thank you!
Did I answer your question? Mark my post as a solution!
Proud to be a Super User!Hello svk666,
In Eventhouse you normally just transform the data there instead of moving it into Lakehouse or using Spark. Since it runs on the Kusto engine, it already handles ingestion and querying in real time.
You can use materialized views if you need pre-aggregated or continuously updated data, or update policies when you want transformation to happen as data is coming in. KQL functions are useful when you want to reuse logic across queries.
Spark or Lakehouse isn’t really needed here unless you’re doing more batch-style processing or ML work that Eventhouse isn’t meant for.
Docs
Eventhouse overview
Materialized views
Update policy overview
Key point
Eventhouse already supports real-time transformation, so most of the time you don’t need to move the data out unless there’s a specific downstream need.Hi svk666 ,
Thanks for reaching out to the Microsoft fabric community forum.
I would also take a moment to thank Srisakthi and suparnababu8 , for actively participating in the community forum and for the solutions you’ve been sharing in the community forum. Your contributions make a real difference.
I hope the above details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We’re always here to support you.Best Regards,
Community Support TeamHi svk666 ,
I hope the above details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We’re always here to support you.
Best Regards,
Community Support Team
