Problems with eventstream's lakehouse transformation

[DennesTorres]

Hi,

When applying an aggregate to a lakehouse transformation in the eventstream, we can't change the name of the aggregated column:

We can't change that "COUNT_*"

Probably as a result, when we choose to create the table in the lakehouse, the table is not created and the process fails.

The message should not be telling me the table was not found, because I asked the target to create it.

Is this a bug or am I doing something wrong?

Kind Regards,

Dennes

[DennesTorres]

After some tests, I discovered the table was in fact created, but the "strange" field name prevents the table to be acessed through SQL Endpoint.

The eventstream, on the other hand, seems to be making the preview through the SQL Endpoint, so the preview fails with the message I sent before.

Besides that, in the middle of the process I changed the Group By configuration, this table should have an additional field, but eventstream didn't manage neither to drop and create again or to send a meaningful message.

Kind Regards,

Dennes

[DennesTorres · Accepted solution]

Hi,

After additional test, I discovered the name of the columns can be customized by a transformation called "Manage Fields".

On the other hand, if I choose an existing table to insert the ingested data, all the fields from the ingestion automatically override the fields on the existing table, specially if the "Manage Fields" is not manually included to rename the ingested fields to the name of the existing table fields.

This seems very strange, could this be considered a bug?

Kind Regards,

Dennes