Machine Learning in Fabric : Logistic Regression with Probabilites - Output Schema in a ML Model

[Anonymous]

Hi All,

I am fairly new to Fabric and ML in general. I had created a logistic regression model for a binary classification task. The model works fine for Prediction and I got my desired result. But as business needs changed, I also wanted to get the probabilities of each class. I used predict_proba method to get the probabilites for the 2 classes. This works on the test data. I get predictions as well as their probabilities.  I saved the experiment as a ML model (using the apply this model in ML Wizard) and followed the steps:

1. Select source data for scoring
2. Map the data correctly to my ML model's inputs
3. Specify the destination for my model's outputs
4. Create a notebook that uses PREDICT to generate and store prediction results as a delta table to the Lakehouse

But, I only get predictions in my delta table and not the probabilities. Is there a way i can also get probabilities? 

PS: I followed the instructions on this link from : https://learn.microsoft.com/en-us/fabric/data-science/model-scoring-predict#use-a-guided-ui-experien...

Another thing i noticed was on the output schema of the experiment and the model, the datatype is int 32, which is right for predictions but should be an array of size [-1,2] for the probabilities.

I am also attaching the link to my notebook just in case: Notebook. 

Thanks,

[DavidDalley66]

Hy I was playing around with logistic regression in Fabric notebooks too, and totally get what you're aiming for. If you're looking to output probabilities instead of just binary predictions, try using predict proba from sklearn. It gives you the actual probabilities for each class instead of just a 0 or 1.Helped me a ton when I needed more nuance in the results!

[DavidDalley66]

Hey! I was playing around with logistic regression in Fabric notebooks too, and totally get what you're aiming for. If you're looking to output probabilities instead of just binary predictions, try using predict_proba() from sklearn. It gives you the actual probabilities for each class instead of just a 0 or 1.Helped me a ton when I needed more nuance in the results!
You can check out the official docs here:
Scikit-learn Logistic Regression Docs

And you can also on my website which I made.

[Anonymous]

Hi @Anonymous ,

Please reach out to our support team to gain deeper insights and explore potential solutions. It's highly recommended that you reach out to our support team. Their expertise will be invaluable in suggesting the most appropriate approach.

Please go ahead and raise a support ticket to reach our support team:

https://support.fabric.microsoft.com/support

After creating a Support ticket please provide the ticket number as it would help us to track for more information.

[Anonymous]

Hi, I have raised a support ticket for this issue. Hereis the case number : 2406290050000495