environment

25 Topics

How are organizations using Microsoft Fabric for end-to-end Machine Learning pipelines?
I'm curious how companies are implementing complete ML workflows in Microsoft Fabric. Do you perform: Data ingestion Feature engineering Model training Model deployment Monitoring entirely inside Fabric, or do you combine Fabric with Azure ML, Databricks, or other platforms? I'd love to learn from real production experiences.
Solved
binitafulpagare
12 days ago Place Data Science
271Views
3likes
5Comments
Fabrics
Could anyone help me understanding the all system of Fabrics? Actually, I am beginner and so interested in this category, I will be so thankful to the person who guides me through it..
Solved
Muhammadluqman
18 days ago Place Data Science
180Views
1like
3Comments
Git Workflow
Hi everyone, Version control is essential for collaborative analytics projects. For teams using Git with Microsoft Fabric: How do you organize repositories? How do you manage branching strategies? Have you integrated Fabric with CI/CD pipelines? What challenges did you face when multiple developers worked on the same project? I'd appreciate hearing how your development workflow has evolved over time. Looking forward to your recommendations.
Solved
binitafulpagare
30 days ago Place Data Science
142Views
0likes
2Comments
Data Science/MLOps in AML vs MS Fabrics
Hi everyone, could someone explain the key differences between implementing MLOps in Azure Machine Learning Studio versus Microsoft Fabric? I am looking to understand the advantages and disadvantages of each. Additionally, which platform is generally preferred for standard data science workflows, and why?
Solved
AlAmeenN
2 months ago Place Data Science
1.6KViews
4likes
7Comments
Update Built-in libraries in new Environment
Hi everyone, I am working in notebooks in Fabric, and I want to update a built-in library to a newer version, by creating a new environment and installing the newer version through PyPI. However, when I start a session with the new notebook, it still uses the same version as in the public library (transformers 4.37.2 instead of 4.49.0). I have also tried installing it through %pip and !pip install. This works in the current session but does not install when I create a new version. I have been given admin rights, but that does not help either. Has anyone experienced this?
Solved
Anonymous
5 months ago Place Data Science
24KViews
3likes
23Comments
Issue with SparkSQL Schema Behavior in Fabric
Hello Fabric Community, I am reaching out to see if anyone else is encountering a similar issue with SparkSQL behavior in Fabric. I have noticed a change in how SparkSQL interacts with schemas. A few days ago, I executed the following script without any issues: USE SCHEMA RawStore; CREATE OR REPLACE TEMPORARY VIEW castTypes AS SELECT CAST(FLOOR(toto) AS STRING) AS toto_r, CAST(FLOOR(rara) AS STRING) AS rara_t, CAST(FLOOR(rite) AS STRING) AS rite_ligne, CAST(FLOOR(vvsf) AS STRING) AS vvsf_fiche FROM mvst; SELECT * FROM castTypes LIMIT 1000; This worked perfectly fine. However, I am now receiving the following error: [SCHEMA_NOT_FOUND] The schema `rawstore` cannot be found. Verify the spelling and correctness of the schema and catalog. If you did not qualify the name with a catalog, verify the current_schema() output, or qualify the name with the correct catalog. To tolerate the error on drop use DROP SCHEMA IF EXISTS. Interestingly, when I prefix the schema name with the workspace and lakehouse name, like workspace.lakehouse.rawstore, like USE SCHEMA workspace.lakehouse.rawstore; CREATE OR REPLACE TEMPORARY VIEW castTypes AS SELECT CAST(FLOOR(toto) AS STRING) AS toto_r, CAST(FLOOR(rara) AS STRING) AS rara_t, CAST(FLOOR(rite) AS STRING) AS rite_ligne, CAST(FLOOR(vvsf) AS STRING) AS vvsf_fiche FROM mvst; SELECT * FROM castTypes LIMIT 1000; the script works again. This indicates that there may be a change in schema context management in SparkSQL within Fabric. Despite my lakehouse being set up with schema options enabled, and the schemas existing properly within my lakehouse, I am puzzled by this change in behavior. Has anyone else faced this issue? If so, how did you resolve it? Any insights or suggestions would be greatly appreciated! Thank you for your help!
Solved
Anonymous
8 months ago Place Data Science
2.7KViews
0likes
3Comments
ultralytics libGL error
Hi I am trying to load cv2 package in fabrics notebook and the error is: ImportError: libGL.so.1: cannot open shared object file: No such file or directory I've tried to install both opencv-python and opencv-python-headless, but neither of them works.
Solved
sliu736
8 months ago Place Data Science
16KViews
1like
6Comments
Custom enviroment notebook
Hi community, I have a custom R notebook development by external users, but then notebook install packages which are not available in "Built-in libraries" (R), if it runs in workspace its works fine, (the notebook download and install the packages). the subject is the libraries are used in several notebooks, In the notebook there is option called it "enviroment" so I understand is possible to create a new enviroment with the libraries "used in the notebook"; so L try to create a new enviroment with custom libraries then the user in the notebook it'll the new enviroment (don´t install again 🙂 ) My questions are: 1.- is this possible to create a new enviroment and set as an option in the "enviroment" option inside notebook? 2.- If the last answer is "YES", best practices to configure dependencies between the new libraries and set up? 3.- How to mantaince the new enviroment for futures updates of libraries? 4.- another options are welcome.. 🙂 thanks in advance
Solved
Peter_23
11 months ago Place Data Science
2.8KViews
1like
2Comments
SQL Query SM in notebook
hi comunnity, i n this time, I try to write a query in SQL to SM by notebook, I know the "explore" option to get the matrix data, and "write DAX queries" are useful tools to query data, but my user it's more comfortable with SQL syntax. So I decided to use notebook to query data from SM using SQL syntax. (sempy ) The first question is: it its possible to query as example: ? to SM 🙂 %%sql SELECT YEAR(OrderDate) AS OrderYear, SUM((UnitPrice * Quantity) + Tax) AS GrossRevenue FROM salesorders GROUP BY YEAR(OrderDate) ORDER BY OrderYear; The next step, I'm tryting to use the library, but dont work . 😞 The SM is a power bi dataset, so I'm using the sempy library. I have a test. In the notebook. I have the next code. import sempy.fabric as fabric dataset = "SM-example matrix filtered by column" # Enter the name or ID of the semantic model workspace = "TEst" # Enter the workspace name or ID in which the semantic model exists<p> <li-code lang="markup">fabric.list_measures(dataset) so It's fine, but the next code, It trigger a expection. # convert to a spark dataframe so that can use select syntax sparkdf = spark.createDataFrame(fabricdf) ValueError: Some of types cannot be determined after inferring Why the valueerror ? 😞 and the next code should be: daxdf=sparkdf.select("Measure Name","Measure Expression").where(sparkdf["Measure Name"]=="Measure") display(daxdf) thanks in advance.
Solved
Peter_23
11 months ago Place Data Science
3.1KViews
0likes
3Comments
Machine Learning in Fabric : Logistic Regression with Probabilites - Output Schema in a ML Model
Hi All, I am fairly new to Fabric and ML in general. I had created a logistic regression model for a binary classification task. The model works fine for Prediction and I got my desired result. But as business needs changed, I also wanted to get the probabilities of each class. I used predict_proba method to get the probabilites for the 2 classes. This works on the test data. I get predictions as well as their probabilities. I saved the experiment as a ML model (using the apply this model in ML Wizard) and followed the steps: 1. Select source data for scoring 2. Map the data correctly to my ML model's inputs 3. Specify the destination for my model's outputs 4. Create a notebook that uses PREDICT to generate and store prediction results as a delta table to the Lakehouse But, I only get predictions in my delta table and not the probabilities. Is there a way i can also get probabilities? PS: I followed the instructions on this link from : https://learn.microsoft.com/en-us/fabric/data-science/model-scoring-predict#use-a-guided-ui-experience Another thing i noticed was on the output schema of the experiment and the model, the datatype is int 32, which is right for predictions but should be an array of size [-1,2] for the probabilities. I am also attaching the link to my notebook just in case: Notebook. Thanks,
Anonymous
1 year ago Place Data Science
3.5KViews
2likes
5Comments