topic How to load a local LLM into MS Fabric? in Data Science

How to load a local LLM into MS Fabric?

supri03 — Tue, 05 Mar 2024 04:08:30 GMT

Hi all,

I have LLAMA 2 LLM locally on my computer, I need to upload that into Fabric and use it to Query on some data I have in the lakehouse. Is this possible to do in Fabric? If so How can I achieve it?

Thanks a lot !

Re: How to load a local LLM into MS Fabric?

Anonymous — Tue, 05 Mar 2024 04:53:19 GMT

Hi supri03
Thanks for using Fabric Community.
Microsoft Fabric and SynapseML provide a powerful set of LLM tools and technologies that enable seamless integration and analysis of data, making it possible to extract meaningful insights from unstructured documents like PDFs.
Please refer to these links for more details :
https://blog.fabric.microsoft.com/en-us/blog/unleashing-the-power-of-synapseml-and-microsoft-fabric-a-guide-to-qa-on-pdf-documents-2?ft=All
Harness the Power of LangChain in Microsoft Fabric for Advanced Document Summarization | Microsoft Fabric Blog | Microsoft Fabric
https://medium.com/@hitesh.hinduja/exploring-microsoft-fabric-fabric-spark-azure-openai-and-langchain-a-hands-on-case-study-26ae949f5977

Hope this helps. Please let me know if you have any further questions.

Re: How to load a local LLM into MS Fabric?

supri03 — Tue, 05 Mar 2024 05:07:13 GMT

Thanks for your response.

My goal is to implement the following:
1. I have data in OneLake
2. I'd like to use Langchain and OpenSource LLM (Without API Keys and Azure OpenAI or OpenAI) to generate a SQL Prompt and it's table ouput , So how can I achieve this without the use of Azure OpenAI or OpenAI services? Is there another way?
Thanks

Re: How to load a local LLM into MS Fabric?

Anonymous — Tue, 05 Mar 2024 05:28:23 GMT

Hi supri03
Just for understanding I would like to know why dont you want to use Azure Open AI or Open AI? We have native LLM endpoints in Fabric so you don't need to bring any keys and you can still use Azure OpenAI APIs.

Re: How to load a local LLM into MS Fabric?

supri03 — Tue, 05 Mar 2024 05:39:23 GMT

We don't have Azure subscription right now. So is there an alternate way to achieve this ?

Re: How to load a local LLM into MS Fabric?

Anonymous — Tue, 05 Mar 2024 07:13:07 GMT

Hi supri03
Apologies. For now this is the possible way to load a local LLM into MS Fabric.
Use Azure AI services in Fabric - Microsoft Fabric | Microsoft Learn
Overview of Copilot in Fabric and Power BI (preview) - Microsoft Fabric | Microsoft Learn

Re: How to load a local LLM into MS Fabric?

Anonymous — Wed, 06 Mar 2024 17:56:54 GMT

Hi supri03
We haven’t heard from you on the last response and was just checking back to see if your query got resolved. Otherwise, will respond back with the more details and we will try to help.
Thanks

Re: How to load a local LLM into MS Fabric?

Anonymous — Fri, 08 Mar 2024 20:18:16 GMT

Hi @supri03
We haven’t heard from you on the last response and was just checking back to see if your query got resolved. Otherwise, will respond back with the more details and we will try to help.
Thanks