Forum Discussion

Kelderic's avatar
Kelderic
Icon for Helper I rankHelper I
1 year ago
Solved

Error In EventStream, Ingested Integer is Outputting as Blank

Hi Everyone,   I have a very simple data structure being pushed into an EventStream. One of the fields is a boolean, which the ES is initially treating as a 0, and classifying as a integer. I first...
eventstream
  • svelde's avatar
    svelde
    1 year ago

    Hello Kelderic 

     

    thanks you for reaching out on this community forum.

     

    The source is the Eventstream is an EventHub living in Azure? Or is this a custom source? Can you share an example of the message you send to the Eventstream? 

     

    Looking at the error, it seems you are forwarding the messages to an Eventhouse KQL database table.

    The 'empty' column is 'null' and the Eventhouse ingestion is complaining.

     

    If you are not using the content of the message body in the Eventstream and just forward each incoming message, consider using the Eventhouse medallion architecture as seen in this post.

     

    You can ingest Eventstream messages in a KQL database table using a dynamic column so the original messages is untouched by the Eventstream.

     

    From there, using Table update policies, you can transform the raw messages into typed columns just the way you prefer the most.

     

     
     
     
     
svelde's avatar
svelde
Icon for Super User rankSuper User
1 year ago

Hello Kelderic 

 

thanks you for reaching out on this community forum.

 

The source is the Eventstream is an EventHub living in Azure? Or is this a custom source? Can you share an example of the message you send to the Eventstream? 

 

Looking at the error, it seems you are forwarding the messages to an Eventhouse KQL database table.

The 'empty' column is 'null' and the Eventhouse ingestion is complaining.

 

If you are not using the content of the message body in the Eventstream and just forward each incoming message, consider using the Eventhouse medallion architecture as seen in this post.

 

You can ingest Eventstream messages in a KQL database table using a dynamic column so the original messages is untouched by the Eventstream.

 

From there, using Table update policies, you can transform the raw messages into typed columns just the way you prefer the most.

 

 
 
 
 
Kelderic's avatar
Kelderic
Icon for Helper I rankHelper I
1 year ago

Here is an example of the JSON being sent:

 

{"sourceTableName":"SLRmas","sourceUniqueID":"R000000001","destinationSystem":"Salesforce","destinationTableName":"Order","destinationUniqueID":"801TN000009aK5WYAU","success":true,"errorMessage":""}

 

I'm wondering if maybe it's the fact that it's a true vs a 1.

 

Edit: That was it. I just changed the input JSON from true|false to an actual 1|0, and it works. It's interesting though, that there is no boolean option in EventStreams. Apparently the ES interprets a boolean as an Int64, but then it fails silently after pushing into the EH. It also incorrectly shows as a 1 or 0 in the preview. Seems like a bug in Fabric.