How are teams using AI agents to automate data science workflows?
I am exploring how AI agents can support modern data science workflows by automating repetitive tasks and improving collaboration between data teams. Some potential use cases: Automating data exploration and preprocessing steps Generating insights from datasets using natural language queries Assisting with feature engineering and model evaluation Triggering ML pipelines based on business events Monitoring model performance and identifying data quality issues I would like to understand how data science teams are combining Microsoft Fabric, notebooks, ML workflows, and AI agents in real-world projects. What approaches, tools, or architecture patterns are you using for AI-assisted data science workflows?Certificate validation error between Fabric and Huggingface
Hi, I'm tyring to download a couple of transformer/sentence-transformer model weights into a notebook and I keep running into certificate mismatch issues between fabric and huggingface. I was wondering if anyone else has ran into this issue or if this is a known issue that Microsoft plans on handling in the very near future? Environment Microsoft Fabric Notebook (PySpark) Python 3.11 PyTorch 2.2.1 Canada tenant sentence-transformers==2.7.0 successfully installed What works Package installation succeeds %pip install sentence-transformers==2.7.0 Import succeeds from sentence_transformers import SentenceTransformer version check succeeds import torch import sentence_transformers print(torch.__version__) print(sentence_transformers.__version__) What fails from sentence_transformers import SentenceTransformer model = SentenceTransformer("all-MiniLM-L6-v2") Or Observations The notebook appears able to reach Hugging Face and download metadata files. For example, the following files are successfully cached: config.json config_sentence_transformers.json modules.json README.md sentence_bert_config.json However, the model weight file never downloads. The Hugging Face cache contains *.incomplete with a file size of 0 MB. Error The relevant error message appears to be "No CA certificates were loaded from the system" Is downloading Hugging Face model weights from Fabric notebooks currently supported? If so, is there a known issue with certificate validation? Is there a recommended workaround? If not. then is there a recommended approach for generating semantic text embeddings within Fabric notebooks utilizing some kind of ANN based algorithm instead of classic techniques such as TF-IDF or SVD followed by some type of clustering? The goal of this notebook is semantic clustering of approximately 500 short "type" descriptions for taxonomy development. Thanks!
Custom enviroment notebook
Hi community, I have a custom R notebook development by external users, but then notebook install packages which are not available in "Built-in libraries" (R), if it runs in workspace its works fine, (the notebook download and install the packages). the subject is the libraries are used in several notebooks, In the notebook there is option called it "enviroment" so I understand is possible to create a new enviroment with the libraries "used in the notebook"; so L try to create a new enviroment with custom libraries then the user in the notebook it'll the new enviroment (don´t install again 🙂 ) My questions are: 1.- is this possible to create a new enviroment and set as an option in the "enviroment" option inside notebook? 2.- If the last answer is "YES", best practices to configure dependencies between the new libraries and set up? 3.- How to mantaince the new enviroment for futures updates of libraries? 4.- another options are welcome.. 🙂 thanks in advance
