How to load a local LLM into MS Fabric?

[supri03]

Hi all,

I have LLAMA 2 LLM locally on my computer, I need to upload that into Fabric and use it to Query on some data I have in the lakehouse. Is this possible to do in Fabric? If so How can I achieve it? 

Thanks a lot !

[Anonymous]

Hi @supri03 
Thanks for using Fabric Community.
Microsoft Fabric and SynapseML provide a powerful set of LLM tools and technologies that enable seamless integration and analysis of data, making it possible to extract meaningful insights from unstructured documents like PDFs.
Please refer to these links for more details :
https://blog.fabric.microsoft.com/en-us/blog/unleashing-the-power-of-synapseml-and-microsoft-fabric-...
Harness the Power of LangChain in Microsoft Fabric for Advanced Document Summarization | Microsoft F...
https://medium.com/@hitesh.hinduja/exploring-microsoft-fabric-fabric-spark-azure-openai-and-langchai...

Hope this helps. Please let me know if you have any further questions.

[supri03]

Thanks for your response. 

My goal is to implement the following:
1. I have data in OneLake
2. I'd like to use Langchain and OpenSource LLM (Without API Keys and Azure OpenAI or OpenAI) to generate a SQL Prompt and it's table ouput , So how can I achieve this without the use of Azure OpenAI or OpenAI services? Is there another way?
Thanks

[Anonymous]

Hi @supri03 
Just for understanding I would like to know why dont you want to use Azure Open AI or Open AI?  We have native LLM endpoints in Fabric so you don't need to bring any keys and you can still use Azure OpenAI APIs.

[supri03]

We don't have Azure subscription right now. So is there an alternate way to achieve this ?

[Anonymous]

Hi @supri03 
Apologies. For now this is the possible way to load a local LLM into MS Fabric. 
Use Azure AI services in Fabric - Microsoft Fabric | Microsoft Learn
Overview of Copilot in Fabric and Power BI (preview) - Microsoft Fabric | Microsoft Learn

[Anonymous]

Hi @supri03 
We haven’t heard from you on the last response and was just checking back to see if your query got resolved. Otherwise, will respond back with the more details and we will try to help.
Thanks

[Anonymous]

Hi @supri03 
We haven’t heard from you on the last response and was just checking back to see if your query got resolved. Otherwise, will respond back with the more details and we will try to help.
Thanks