environment

26 Topics

Certificate validation error between Fabric and Huggingface
Hi, I'm tyring to download a couple of transformer/sentence-transformer model weights into a notebook and I keep running into certificate mismatch issues between fabric and huggingface. I was wondering if anyone else has ran into this issue or if this is a known issue that Microsoft plans on handling in the very near future? Environment Microsoft Fabric Notebook (PySpark) Python 3.11 PyTorch 2.2.1 Canada tenant sentence-transformers==2.7.0 successfully installed What works Package installation succeeds %pip install sentence-transformers==2.7.0 Import succeeds from sentence_transformers import SentenceTransformer version check succeeds import torch import sentence_transformers print(torch.__version__) print(sentence_transformers.__version__) What fails from sentence_transformers import SentenceTransformer model = SentenceTransformer("all-MiniLM-L6-v2") Or Observations The notebook appears able to reach Hugging Face and download metadata files. For example, the following files are successfully cached: config.json config_sentence_transformers.json modules.json README.md sentence_bert_config.json However, the model weight file never downloads. The Hugging Face cache contains *.incomplete with a file size of 0 MB. Error The relevant error message appears to be "No CA certificates were loaded from the system" Is downloading Hugging Face model weights from Fabric notebooks currently supported? If so, is there a known issue with certificate validation? Is there a recommended workaround? If not. then is there a recommended approach for generating semantic text embeddings within Fabric notebooks utilizing some kind of ANN based algorithm instead of classic techniques such as TF-IDF or SVD followed by some type of clustering? The goal of this notebook is semantic clustering of approximately 500 short "type" descriptions for taxonomy development. Thanks!
Solved
pborah
3 days ago Place Data Science
23Views
0likes
1Comment
How are organizations using Microsoft Fabric for end-to-end Machine Learning pipelines?
I'm curious how companies are implementing complete ML workflows in Microsoft Fabric. Do you perform: Data ingestion Feature engineering Model training Model deployment Monitoring entirely inside Fabric, or do you combine Fabric with Azure ML, Databricks, or other platforms? I'd love to learn from real production experiences.
Solved
binitafulpagare
10 days ago Place Data Science
312Views
4likes
6Comments
Fabrics
Could anyone help me understanding the all system of Fabrics? Actually, I am beginner and so interested in this category, I will be so thankful to the person who guides me through it..
Solved
Muhammadluqman
1 month ago Place Data Science
184Views
1like
3Comments
Git Workflow
Hi everyone, Version control is essential for collaborative analytics projects. For teams using Git with Microsoft Fabric: How do you organize repositories? How do you manage branching strategies? Have you integrated Fabric with CI/CD pipelines? What challenges did you face when multiple developers worked on the same project? I'd appreciate hearing how your development workflow has evolved over time. Looking forward to your recommendations.
Solved
binitafulpagare
1 month ago Place Data Science
146Views
0likes
2Comments
Data Science/MLOps in AML vs MS Fabrics
Hi everyone, could someone explain the key differences between implementing MLOps in Azure Machine Learning Studio versus Microsoft Fabric? I am looking to understand the advantages and disadvantages of each. Additionally, which platform is generally preferred for standard data science workflows, and why?
Solved
AlAmeenN
2 months ago Place Data Science
1.6KViews
4likes
7Comments
Update Built-in libraries in new Environment
Hi everyone, I am working in notebooks in Fabric, and I want to update a built-in library to a newer version, by creating a new environment and installing the newer version through PyPI. However, when I start a session with the new notebook, it still uses the same version as in the public library (transformers 4.37.2 instead of 4.49.0). I have also tried installing it through %pip and !pip install. This works in the current session but does not install when I create a new version. I have been given admin rights, but that does not help either. Has anyone experienced this?
Solved
Anonymous
6 months ago Place Data Science
24KViews
3likes
23Comments
Issue with SparkSQL Schema Behavior in Fabric
Hello Fabric Community, I am reaching out to see if anyone else is encountering a similar issue with SparkSQL behavior in Fabric. I have noticed a change in how SparkSQL interacts with schemas. A few days ago, I executed the following script without any issues: USE SCHEMA RawStore; CREATE OR REPLACE TEMPORARY VIEW castTypes AS SELECT CAST(FLOOR(toto) AS STRING) AS toto_r, CAST(FLOOR(rara) AS STRING) AS rara_t, CAST(FLOOR(rite) AS STRING) AS rite_ligne, CAST(FLOOR(vvsf) AS STRING) AS vvsf_fiche FROM mvst; SELECT * FROM castTypes LIMIT 1000; This worked perfectly fine. However, I am now receiving the following error: [SCHEMA_NOT_FOUND] The schema `rawstore` cannot be found. Verify the spelling and correctness of the schema and catalog. If you did not qualify the name with a catalog, verify the current_schema() output, or qualify the name with the correct catalog. To tolerate the error on drop use DROP SCHEMA IF EXISTS. Interestingly, when I prefix the schema name with the workspace and lakehouse name, like workspace.lakehouse.rawstore, like USE SCHEMA workspace.lakehouse.rawstore; CREATE OR REPLACE TEMPORARY VIEW castTypes AS SELECT CAST(FLOOR(toto) AS STRING) AS toto_r, CAST(FLOOR(rara) AS STRING) AS rara_t, CAST(FLOOR(rite) AS STRING) AS rite_ligne, CAST(FLOOR(vvsf) AS STRING) AS vvsf_fiche FROM mvst; SELECT * FROM castTypes LIMIT 1000; the script works again. This indicates that there may be a change in schema context management in SparkSQL within Fabric. Despite my lakehouse being set up with schema options enabled, and the schemas existing properly within my lakehouse, I am puzzled by this change in behavior. Has anyone else faced this issue? If so, how did you resolve it? Any insights or suggestions would be greatly appreciated! Thank you for your help!
Solved
Anonymous
8 months ago Place Data Science
2.7KViews
0likes
3Comments
ultralytics libGL error
Hi I am trying to load cv2 package in fabrics notebook and the error is: ImportError: libGL.so.1: cannot open shared object file: No such file or directory I've tried to install both opencv-python and opencv-python-headless, but neither of them works.
Solved
sliu736
9 months ago Place Data Science
16KViews
1like
6Comments
Custom enviroment notebook
Hi community, I have a custom R notebook development by external users, but then notebook install packages which are not available in "Built-in libraries" (R), if it runs in workspace its works fine, (the notebook download and install the packages). the subject is the libraries are used in several notebooks, In the notebook there is option called it "enviroment" so I understand is possible to create a new enviroment with the libraries "used in the notebook"; so L try to create a new enviroment with custom libraries then the user in the notebook it'll the new enviroment (don´t install again 🙂 ) My questions are: 1.- is this possible to create a new enviroment and set as an option in the "enviroment" option inside notebook? 2.- If the last answer is "YES", best practices to configure dependencies between the new libraries and set up? 3.- How to mantaince the new enviroment for futures updates of libraries? 4.- another options are welcome.. 🙂 thanks in advance
Solved
Peter_23
11 months ago Place Data Science
2.9KViews
1like
2Comments
SQL Query SM in notebook
hi comunnity, i n this time, I try to write a query in SQL to SM by notebook, I know the "explore" option to get the matrix data, and "write DAX queries" are useful tools to query data, but my user it's more comfortable with SQL syntax. So I decided to use notebook to query data from SM using SQL syntax. (sempy ) The first question is: it its possible to query as example: ? to SM 🙂 %%sql SELECT YEAR(OrderDate) AS OrderYear, SUM((UnitPrice * Quantity) + Tax) AS GrossRevenue FROM salesorders GROUP BY YEAR(OrderDate) ORDER BY OrderYear; The next step, I'm tryting to use the library, but dont work . 😞 The SM is a power bi dataset, so I'm using the sempy library. I have a test. In the notebook. I have the next code. import sempy.fabric as fabric dataset = "SM-example matrix filtered by column" # Enter the name or ID of the semantic model workspace = "TEst" # Enter the workspace name or ID in which the semantic model exists<p> <li-code lang="markup">fabric.list_measures(dataset) so It's fine, but the next code, It trigger a expection. # convert to a spark dataframe so that can use select syntax sparkdf = spark.createDataFrame(fabricdf) ValueError: Some of types cannot be determined after inferring Why the valueerror ? 😞 and the next code should be: daxdf=sparkdf.select("Measure Name","Measure Expression").where(sparkdf["Measure Name"]=="Measure") display(daxdf) thanks in advance.
Solved
Peter_23
1 year ago Place Data Science
3.1KViews
0likes
3Comments