Forum Discussion
Machine Learning in Fabric : Logistic Regression with Probabilites - Output Schema in a ML Model
Hi All,
I am fairly new to Fabric and ML in general. I had created a logistic regression model for a binary classification task. The model works fine for Prediction and I got my desired result. But as business needs changed, I also wanted to get the probabilities of each class. I used predict_proba method to get the probabilites for the 2 classes. This works on the test data. I get predictions as well as their probabilities. I saved the experiment as a ML model (using the apply this model in ML Wizard) and followed the steps:
1. Select source data for scoring
2. Map the data correctly to my ML model's inputs
3. Specify the destination for my model's outputs
4. Create a notebook that uses PREDICT to generate and store prediction results as a delta table to the Lakehouse
But, I only get predictions in my delta table and not the probabilities. Is there a way i can also get probabilities?
PS: I followed the instructions on this link from : https://learn.microsoft.com/en-us/fabric/data-science/model-scoring-predict#use-a-guided-ui-experience
Another thing i noticed was on the output schema of the experiment and the model, the datatype is int 32, which is right for predictions but should be an array of size [-1,2] for the probabilities.
I am also attaching the link to my notebook just in case: Notebook.
Thanks,
5 Replies
Hi Anonymous ,
Please reach out to our support team to gain deeper insights and explore potential solutions. It's highly recommended that you reach out to our support team. Their expertise will be invaluable in suggesting the most appropriate approach.
Please go ahead and raise a support ticket to reach our support team:
https://support.fabric.microsoft.com/support
After creating a Support ticket please provide the ticket number as it would help us to track for more information.
Hey! I was playing around with logistic regression in Fabric notebooks too, and totally get what you're aiming for. If you're looking to output probabilities instead of just binary predictions, try using predict_proba() from sklearn. It gives you the actual probabilities for each class instead of just a 0 or 1.Helped me a ton when I needed more nuance in the results!
You can check out the official docs here:
Scikit-learn Logistic Regression DocsAnd you can also on my website which I made.
Hy I was playing around with logistic regression in Fabric notebooks too, and totally get what you're aiming for. If you're looking to output probabilities instead of just binary predictions, try using predict proba from sklearn. It gives you the actual probabilities for each class instead of just a 0 or 1.Helped me a ton when I needed more nuance in the results!
Nice one
