Forum Discussion
Using aifunc in notebook to extract csv data
Hi Ira_27 ,
The behavior you're describing is expected. By default, ai.extract returns a list for each label, even if only one value is found. That's why you see ["ABC123"] instead of ABC123 — it's not a bug, it's how it works internally.
The fix is max_items=1 (https://learn.microsoft.com/en-us/fabric/data-science/ai-functions/pyspark/extract?tabs=labels#return) That parameter tells the function to return a scalar instead of a list. Give this a try this:
labels=[
aifunc.ExtractLabel(
"Name",
description="Return only the Name without brackets or quotes.",
max_items=1,
type="string"
),
aifunc.ExtractLabel("Age", max_items=1, type="integer"),
aifunc.ExtractLabel("Total %", max_items=1, type="number"),
]For other hand If your data starts at row 10, it's worth filtering those rows out before calling ai.extract — otherwise the model will try to extract values from lines that aren't actual data, which can produce empty or incorrect results for those rows.
If my comment helped solve your question, it would be great if you could like the comment and mark it as the accepted solution. It helps others with the same issue and also motivates me to keep contributing.
Thanks a lot, I really appreciate it.
Hello Ira_27
Microsoft clearly documents this about ai.extract in the following:
"The default return type is a list of strings for each label. When max_items isn't specified, multiple matches are returned as a list."
Use ai.extract with PySpark - Microsoft Fabric | Microsoft Learn
Within the function you can use max_items=1 to force the function to return as a single element list, not multiple values. You can then convert the array to a scaler value by doing either element_at(col("Name"), 1) or col("Name")[0].
The fact that the labels are surrounded by quotes is simply the model chosing to return the string in that way. This is a common behaviour. You can trim whitespace and remove wrapping quotes as a final clean up as appropriate.
These are some of the approaches you can use if you're planning to run this in a production capacity.