FUAM Load_Items_E2E Error in Copy Data

[pbi_taken]

Hi.
I'm trying to get the FUAM package to work and after an update I keep getting errors. The first one I have narrowed it down to is the get items part of Load_items. 

My source is the active_items table in FUAM_Lakehouse ( is this correct to begin with?).

But this table can’t load because the column creatorPrincipal because the data type is not supported in Delta format.

Can anyone please tell me what I’m doing wrong?

Thanks!

[pbi_taken · Accepted solution]

Hi all. 
Solution was just to upload the notebook "Deploy_FUAM" and run it again. This solved the issue, so I'm guessing it was something with an update that has been done to this notebook.

Thanks to everyone that helped!

[v-kpoloju-msft]

Hi  @pbi_taken,

Thank you for reaching out and providing the error details. Also thanks @mabdollahi, for his inputs on this issue.

The issue arises from the creatorPrincipal column in your active_items table. This column appears to contain a complex or unsupported data type (such as a nested object or user principal), which is not supported by the Delta format in Lakehouse.

To address this, you will need to exclude the creatorPrincipal column from your data source before the pipeline processes it.

Create a new version of your table (or view) that removes the problematic column:

CREATE OR REPLACE TABLE active_items_clean AS

SELECT * EXCEPT (creatorPrincipal)

FROM active_items;

Update your Copy Data / Load_Items_E2E pipeline to point to active_items_clean instead of the original active_items table.

If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.

Thank you for using Microsoft Community Forum.

[v-kpoloju-msft]

Hi @pbi_taken,

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

[mabdollahi]

Hi @pbi_taken 
The column was added in 2025-03-01 as per https://github.com/GT-Analytics/fuam-basic/releases#:~:text=New%20column%20in,item%20more%20efficien... 

How are you loading the data? I see in copy data, can you consider Notebook instead?

If you are loading the data in notebook, then you can simply remove the column before loading to start working on the table.

df = spark.read.table("FUAM_Lakehouse.active_items")
df = df.drop("creatorPrincipal")  
df.write.format("delta").saveAsTable("TargetTable")

You can also try to convert the column in a text format before writing to another table.

I hope this answers your question.

Please feel free to mark it as correct solution if it does and kodus. 🙂 

Regards,

Mehrdad Abdollahi

[pbi_taken]

but this is an out of the box solution and probably should not need to do my own editing of this? So it's either outdatet or I have edited the connection and made an error

[mabdollahi]

@pbi_taken as you know it is built by community experts, have you logged it as an error under the comitted version on GitHub? As you might know it was written by community.