Error In EventStream, Ingested Integer is Outputting as Blank

[Kelderic]

Hi Everyone,

I have a very simple data structure being pushed into an EventStream. One of the fields is a boolean, which the ES is initially treating as a 0, and classifying as a integer. I first connected it directly to an EH, and then tried a ManageFields in between with all fields, but the same thing happens. In the EH, the field comes over as blank.

I looked into the Runtime logs and found this error:

"value": "Output is currently experiencing transient errors. Retrying...- KQL Database Output Error for KustoClient#2[db: FDI_EH_Logging, table: PerRecord]. Error: A permanent error occurred while attempting to ingest: 'Stream' via streaming ingestion. Error: '<ExtendedException.MessageEx: Null exception object>'\r\n- A permanent error occurred while attempting to ingest: 'Stream' via streaming ingestion. Error: '<ExtendedException.MessageEx: Null exception object>'\r\n- <ExtendedException.MessageEx: Null exception object>\r\n\r\n",

When I click on the Ingestion, I get the preview here, with the 1s:

When I click on the output, I get the following preview, with blanks:

Edit: The mapping is here: Nothing is changing, I just clicked on Add All Fields:

[svelde · Accepted solution]

Hello @Kelderic 

thanks you for reaching out on this community forum.

The source is the Eventstream is an EventHub living in Azure? Or is this a custom source? Can you share an example of the message you send to the Eventstream? 

Looking at the error, it seems you are forwarding the messages to an Eventhouse KQL database table.

The 'empty' column is 'null' and the Eventhouse ingestion is complaining.

If you are not using the content of the message body in the Eventstream and just forward each incoming message, consider using the Eventhouse medallion architecture as seen in this post.

You can ingest Eventstream messages in a KQL database table using a dynamic column so the original messages is untouched by the Eventstream.

From there, using Table update policies, you can transform the raw messages into typed columns just the way you prefer the most.

[Kelderic]

Here is an example of the JSON being sent:

{"sourceTableName":"SLRmas","sourceUniqueID":"R000000001","destinationSystem":"Salesforce","destinationTableName":"Order","destinationUniqueID":"801TN000009aK5WYAU","success":true,"errorMessage":""}

I'm wondering if maybe it's the fact that it's a true vs a 1.

Edit: That was it. I just changed the input JSON from true|false to an actual 1|0, and it works. It's interesting though, that there is no boolean option in EventStreams. Apparently the ES interprets a boolean as an Int64, but then it fails silently after pushing into the EH. It also incorrectly shows as a 1 or 0 in the preview. Seems like a bug in Fabric.