Forum Discussion

raniasabdullah's avatar
raniasabdullah
Regular Visitor
1 year ago

Eventstream Reading from Event Hub with Missing Columns in Lakehouse Ingestion

I'm using Eventstream to read JSON messages from Azure Event Hub. The data in Eventstream Hub appears correctly after multiple refreshes. However, when I ingest the data into the lakehouse or Eventho...
eventstream
Anonymous's avatar
Anonymous
Not applicable
1 year ago

Hi raniasabdullah ,

 

Please check for violations of the following restrictions:

  • You can't change an existing column type. If you try to map to a column having a different format, you may end up with empty columns.

  • Some data format mappings (Parquet, JSON, and Avro) support simple ingest-time transformations. To apply mapping transformations, create or update a column in the Edit columns window.

    Mapping transformations can be performed on a column of type string or datetime, with the source having data type int or long. Supported mapping transformations are:

    • DateTimeFromUnixSeconds
    • DateTimeFromUnixMilliseconds
    • DateTimeFromUnixMicroseconds
    • DateTimeFromUnixNanoseconds

Ensure that the content-type and headers are correctly set when sending JSON messages to the Event Hub.

For more details, please refer:

Get data from Azure Event Hubs - Microsoft Fabric | Microsoft Learn

Reading-data-from-event-hubs-into-eventstream-and-a-lakehouse-table

 

 

Best Regards,

Neeko Tang

If this post  helps, then please consider Accept it as the solution  to help the other members find it more quickly.