Hi @fabricuser123_ ,
First thing to assume: AI Functions results will never be 100% deterministic. Microsoft's own documentation includes the warning # This code uses AI. Always review output for mistakes in every code example — that's not decorative, it's a real acknowledgment of expected model behavior.
That said, there are two approaches depending on what you need:
Scenario A — Validate before trusting results
Microsoft has published evaluation notebooks that measure output quality using LLM-as-a-Judge: a larger model acts as evaluator and computes accuracy, precision, recall and F1 metrics. The workflow is: run the function with an executor model, evaluate with a judge model, identify which predictions need review, and refine labels or configuration. ( https://blog.fabric.microsoft.com/en-us/blog/unlock-insights-from-images-and-pdfs-with-multimodal-su... )
Use this before going to production to establish a quality baseline on your own dataset.
Scenario B — Control quality directly in production
A more robust pattern is adding a Data Quality layer after the AI Function:
- Layer 1: the AI Function generates the label → sentiment_ai column
- Layer 2: a DQ rule flags uncertain cases → needs_review = true when the model returns mixed, the text is very short, or contains negations/irony
There is a documented pattern where, if a second judge model disagrees with the assigned label, the pipeline conditionally triggers a relabeling step. This loop is orchestrated using Fabric Pipelines, which support conditional and iterative control flow. (https://learn.microsoft.com/en-us/fabric/data-science/tutorial-text-classification)
The human factor doesn't disappear — it gets focused on flagged cases, not 100% of the dataset. That's what makes the process sustainable at scale.
If this response was helpful, please give it a like and mark it as a solution — it encourages me to keep contributing and helps other users find answers faster 🙌
