Using aifunc in notebook to extract csv data

[Ira_27]

Hi Microsoft Fabric Community,

I am testing the new AI Functions in Microsoft Fabric notebooks and using df.ai.extract() with aifunc.ExtractLabel() to extract values from a csv file that has comma seperated value where data starts from 10th row.

My code looks like this:

from synapse.ml.spark import aifunc
df = spark.read.text("Files/xxx.csv")
dfcsv = df.ai.extract(
    labels=[
        aifunc.ExtractLabel(
            "Name",
            description="Return only the Name without brackets or quotes."
        ),
        "Name",
        "Age",
        "Total %"
    ],
    input_col="value"
)

The extraction works, but the returned values often come back like:

["ABC123"]
"ABC123"
['ABC123']

Instead of plain text: "ABC123"

My Questions:

1. Is this expected behavior for aifunc.extract()?
2. Is there a parameter or setting to force scalar/plain text output instead of arrays or quoted strings?
3. What is the recommended production approach for cleaning or normalizing these outputs in Fabric notebooks?
4. If multiple values are detected, how does the function decide array vs string return type?

I’d appreciate any best practices or examples from others using AI Functions in Fabric.

Thank you.

[deborshi_nag]

Hello @Ira_27 

Microsoft clearly documents this about ai.extract in the following:

"The default return type is a list of strings for each label. When max_items isn't specified, multiple matches are returned as a list."

Use ai.extract with PySpark - Microsoft Fabric | Microsoft Learn

Within the function you can use max_items=1 to force the function to return as a single element list, not multiple values. You can then convert the array to a scaler value by doing either element_at(col("Name"), 1) or col("Name")[0].

The fact that the labels are surrounded by quotes is simply the model chosing to return the string in that way. This is a common behaviour. You can trim whitespace and remove wrapping quotes as a final clean up as appropriate. 

These are some of the approaches you can use if you're planning to run this in a production capacity.

[arabalca]

Hi @Ira_27 ,

The behavior you're describing is expected. By default, ai.extract returns a list for each label, even if only one value is found. That's why you see ["ABC123"] instead of ABC123 — it's not a bug, it's how it works internally.

The fix is max_items=1 (https://learn.microsoft.com/en-us/fabric/data-science/ai-functions/pyspark/extract?tabs=labels#retur...) That parameter tells the function to return a scalar instead of a list. Give this a try this:

labels=[
        aifunc.ExtractLabel(
            "Name",
            description="Return only the Name without brackets or quotes.",
            max_items=1,
            type="string"
        ),
        aifunc.ExtractLabel("Age", max_items=1, type="integer"),
        aifunc.ExtractLabel("Total %", max_items=1, type="number"),
    ]

For other hand If your data starts at row 10, it's worth filtering those rows out before calling ai.extract — otherwise the model will try to extract values from lines that aren't actual data, which can produce empty or incorrect results for those rows.

If my comment helped solve your question, it would be great if you could like the comment and mark it as the accepted solution. It helps others with the same issue and also motivates me to keep contributing.

Thanks a lot, I really appreciate it.