Idea:
As an engineer responsible for managing and improving Microsoft Fabric Data Agents, I currently have no way to view how users are interacting with the agents I’ve built. While it’s likely that prompt and response data is being captured internally by Microsoft for telemetry purposes, there is no mechanism that makes this data available to those of us maintaining these solutions.
Proposal:
Introduce a feature within Fabric Data Agents that provides engineers with controlled access to interaction logs, including:
User-submitted prompts
Agent responses
Feedback indicators (e.g., thumbs up/down, optional comments)
Outcome metadata, such as whether the interaction was helpful or resulted in follow-up
Why this is critical:
Without access to this data, engineers are operating in the dark. We cannot evaluate how our agents are performing, where they are failing, or what users are actually trying to accomplish. This limits our ability to iterate, improve relevance, and ensure that Fabric Data Agents deliver real value.
Key Benefits:
Understand real-world usage and identify unmet needs
Improve agent behavior by refining instruction sets and context
Proactively address recurring failure patterns
Increase user trust and adoption by continuously tuning responses
Align agent capabilities with actual user workflows
Governance & Privacy:
Access should be controlled by tenant-level permissions and comply with Microsoft’s data governance policies. Logs can be anonymized and filtered to protect sensitive content, with visibility limited to approved roles (e.g., workspace admins or agent developers).
Summary:
Microsoft Fabric needs to provide engineers with visibility into how their Fabric Data Agents are being used. Access to prompt and response history, combined with user feedback, is essential to improving agent quality and delivering successful AI-driven experiences.
jeroenterheerdt
on:
Visual Calculation and Fields Parameter
on:
Deployment Pipeline Can't Perform Comparison and W...
on:
absolutely essential !! able disable or show/hide...
on:
elipsis menus in alphabetical order
ToddChitt
on:
Remove Table List from Pipeline Task for Semantic ...