Copilot for Data Science/powerBI: Customization Options?

[hsn367]

Our team is currently exploring copilot for data science and powerbi and we are interested in understanding the scope of customization available for Copilot to better suit our data science needs.

1. Prompt Engineering: Are there established practices within Fabric for optimizing prompts in Copilot specifically for data science applications? Any tips or examples would be greatly appreciated.

2. Model Fine-Tuning: Is it possible to fine-tune the underlying models of Copilot to better align with our specific needs and problem statements? If so, what are the steps and considerations for initiating such a process?

[hsn367]

Hi @v-yaningy-msft thank you for the support. Here are a few results from my interaction with the copilot for data science. I used copilot in notebooks using magic commands and here is my input prompt:

"In the df_all_data dataframe we have promotion data and our goal is to predict how many employees are gonna get promoted in each market in every month for next one year. Analyze the data and recommend some ML models that can be used on this data"

In session 1, I got this response: "I must decline"

In session 2, when I executed the same cell, I got this response: "I must decline to provide recommendations on ML models without access to the data or more context. Feel free to share more details for assistance"

This is strange, because as per the documentation, it has access to the data loaded in dataframes. I have not changed any data sharing settings as well. Also when I execute following magic command to check data sharing level

%set_sharing_level?

I get this response:  Object `%set_sharing_level` not found.

[v-yaningy-msft · Accepted solution]

Hi, @hsn367 

When you use Copilot, you need to be more precise in describing your needs, such as the tables used, the columns in the tables, etc. We recommend you to watch some videos to get a quick start, here are some links.
https://youtu.be/4hh1e6oy2A4?si=D10m-m89wwUrEysn
https://youtu.be/L3ZC46lAUhI?si=QV1Y191KBVB9Yp5s
https://youtu.be/H3LND_ErY5U?si=lJasF7IjqxwU-2Rt

As for the set_sharing_level command, it's not in Configuration Commands now; it was in the pre-June version, and apparently it's not supported anymore, but the documentation isn't up to date.

Six month ago:
https://youtu.be/4hh1e6oy2A4?si=D10m-m89wwUrEysn  (2:49)

Now: 

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

How to get your questions answered quickly --  How to provide sample data in the Power BI Forum

[v-yaningy-msft]

Hi, @hsn367 

So far, the Copilot does not provide support for PowerBI or direct integration with data science platforms, nor does it support tuning of the Copilot's underlying model. For the time being, the Copilot for Data science supports the following features, and you can go through the corresponding question statements for a more precise answer(Copilot features in the Data Science experience are currently scoped to notebooks).

%%code
Can you generate code for a logistic regression that fits this data?

%%chat
What are some machine learning models that may fit this dataset?

Overview of Copilot for Data Science and Data Engineering in Microsoft Fabric (preview) - Microsoft ...

You can submit an idea for here and the product team will take note of such a need.

Related Link: New tab (powerbi.com)

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

How to get your questions answered quickly --  How to provide sample data in the Power BI Forum