Forum Discussion
Getting json payload from Azure Event Grid / Event Hubs and just save it to the Lakehouse
Hi foks,
Thank you for reaching out to the Microsoft fabric community forum.When event data from Event Grid / Event Hub is ingested into a Lakehouse using streaming paths, Fabric automatically parses the JSON and stores it as a Delta table for analytics. There is not a supported option today to tell Fabric to keep the incoming payload as a raw JSON file in the Lakehouse Files area.
If your requirement is to preserve the events exactly as received raw JSON, you will need to land them outside the default streaming-to-Lakehouse experience.
For example capture the events using a custom ingestion Azure Function, Logic App, or Data Factory and write them directly as JSON files to storage/Lakehouse Files. If the goal is analytics on event or telemetry data, Eventhouse (KQL) is the recommended approach, as it’s designed for this type of time-series data.
Kindly refer to the below documentation links for better understanding:
Add a lakehouse destination to an eventstream - Microsoft Fabric | Microsoft LearnHope this clarifies why the data is getting flattened and what the available options are today.
Regards,
Community Support Team.
Hello foks ,
Welcome to this community forum.
Have you considered forwarding incoming MQTT messages in the Eventhouse?
Check out this blogpost for details.
I ask this because a timeseries is in most cases a better fit for data arriving as event, measurements, immutable telemetry.
the KQL query language is perfect for working with time-based data.
If you still need a (JSON) file for other tooling, you can use a Lakehouse shortcut so the data appears as a table.
If this answer helps you, a thumbs-up or marking it as accepted answer is appreaciated.