Is there any implementation (for ref.) of custom LLM models/Gen AI models hit by Power BI Reports?

[roysampad91]

Is there any implementation (for ref.) of custom LLM models/Gen AI models hit by Power BI Reports?

Understand using Smart Narrative and using the LLM Model API a connection can be estabilished theoritically but has anyone created a solution by connecting a custom LLM Model/Gen AI to Power BI?

Use Case: User uses Smart Narrative to ask a question which is passed to the Custom LLM model and the response is reverted back to the user on the Smart Narrative Visual.

Any reference implementations would be very helpful. Thanks in Advance.

[Ritaf1983 · Accepted solution]

Hi @roysampad91 

Integrating large language models (LLM) with Power BI is currently not very popular, due to several reasons:

• Technical Complexity: Integrating an LLM with Power BI requires APIs, cloud services like Azure or AWS, and sometimes Power Automate or Azure Functions, making it complex and less accessible for many users.
• Performance Costs: LLMs can add latency, which may slow down Power BI reports where users expect quick responses.
• Limited Use Cases: Most Power BI users focus on traditional data analysis, so use cases for LLMs are narrow, mainly suited for scenarios needing detailed narrative explanations or language-based insights.
• Existing Power BI Tools: Built-in features like Smart Narrative, Q&A, and AI Insights often fulfill users' needs, reducing demand for custom LLM models.
• Privacy and Security Concerns: Using LLM models typically involves sending data to external servers, which may be a challenge for organizations with strict privacy requirements.
• API Rate Limits: Some LLM providers have rate limits, which can restrict scalability for large user bases in Power BI.

If you still want to explore LLM integration with Power BI, here are some options:

• Azure OpenAI Service: Use the Azure OpenAI Service API to connect Power BI to an Azure-hosted LLM model: https://learn.microsoft.com/en-us/azure/architecture/solutions/power-platform-scenarios
• Power BI Custom Visuals and REST API: Create a custom visual to interact with an LLM API or set up Power Query to fetch responses directly. Link: https://learn.microsoft.com/en-us/connectors/powerbi/
• Using Power Automate: Power Automate can act as an intermediary between Power BI and the LLM model, triggering a flow that sends queries to the LLM API and returns responses to Power BI: https://learn.microsoft.com/en-us/power-bi/collaborate-share/service-flow-integration
• Power BI and Azure Functions: Use Azure Functions to create a custom API endpoint for handling requests and responses with the LLM model: https://learn.microsoft.com/en-us/azure/architecture/solutions/power-platform-scenarios
  For additional resources:
• Custom Visuals: Explore tools for creating a Power BI custom visual that communicates with your LLM model: https://learn.microsoft.com/en-us/power-bi/create-reports/power-bi-automate-visual
• Connecting Power BI to Azure AI: Microsoft’s blog on integrating Power BI with Azure AI services for a foundation in AI and Power BI interaction: https://learn.microsoft.com/en-us/azure/architecture/solutions/power-platform-scenarios

If this post helps, then please consider Accepting it as the solution to help the other members find it more quickly.