Managing unneeded event columns before to write into a lakehouse destination

[pmscorca]

Hi,

I've an eventstream connected to an Azure Event hubs source.

I need to get more events to put them in a lakehouse as a destination.

Each event is composed of ten columns but I need to consider only 9 columns and I need to discard an unneeded json column that also caused a conversion error.

I've tried to use the Manage fields feature for the lakehouse destination, but it seems that it isn't possible to unconsider the unneeded column.

Now, how could I consider only the needed event columns to write on a lakehouse destination? Thanks

[Anonymous]

Hi @pmscorca ,

In your eventstream you can add a Manage Fields step:

Click here:

Choose the field you want to remove:

Best Regards,
Dino Tao
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[pmscorca]

Hi,

is it possible to remove the source event data in the Event processing editor upstream of Manage fields?

I think that it could be useful to remove the upstream fields in order to avoid to change the related type for unneeded data.

[Anonymous]

Hi @pmscorca ,

You want to remove this one?

Sorry, this tile is currently unavailable for operation.

Best Regards,
Dino Tao

[pmscorca]

Hi, not all but only some data of the events, those the unneeded or undesidered ones.

Thanks

[pmscorca]

Hi, thanks for your reply. I've already tried to use the Manage fields feature.

However, I've solved because the source events have one JSON data with mixed values, some as records and other ones as arrays.

So the issue is related to the event values and not about the Manage fields feauture.