data engineering
369 TopicsUnlink Microsoft Onelake
Hi everyone. I'm facing an issue when I try to unlink my Dataverse Link to Fabric link. I cannot add new tables, cannot unlink, create a new one. When I try to unlink, it shows this: If I go to 'details', it shows this: And if I go to 'manage tables', I face this error: I know that in all these cases I need to contact the support team, but I'd like to give a shot here before. Any guesses of what could be causing that? I tried to link, unlink, I even deleted the workspace and tried to set up in a new one, but it shows that it already exists: "Your tables are already linked with Microsoft Fabric".Solved3.6KViews0likes6CommentsDynamics 365 F&O - Fabric Integration
Hi All, I am trying to integrate Dynamics 365 F&O data to Fabric. I replicated the entities in Dynamics 365 F&O to Dataverse. They come as virtual tables in Datavese. I am trying to use 'Link to Fabric' in Dataverse platform and shortcut in Fabric as well. But those do not replicate the virtual tables that had been brought to Dataverse. Does anyone know how to solve this problem?Solved6.6KViews0likes13Comments"The remote certificate is invalid" error on pipeline Fabric job to copy parquet files from AWS S3
When I try to copy parquet files from AWS S3 bucket with "copy data" job on pipelline function, this error show up: "The file operation is failed. A WebException with status TrustFailure was thrown. The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. The remote certificate is invalid according to the validation procedure. Activity ID: ******" The test connection results OK. The shortcut function works well too. But we need to copy the data from S3 with pipeline function, the shortcut don't help us. We try with connection type "Amazon S3 Compatible" too and we get the same error. Some one can help or are getting this error too? Thanks!1.3KViews0likes2CommentsGraphframes error: ClassNotFoundException
I am trying to use Graphframes library to generate GraphX objects. Output of "%conda list" shows : graphframes 0.6 pyhd3deb0d_1 conda-forge So the package is installed. But when I try to construct a GraphFrame g = GraphFrame(v,edge) I get the following error: Py4JJavaError: An error occurred while calling o5762.loadClass. : java.lang.ClassNotFoundException: org.graphframes.GraphFramePythonAPI What do I need to get this to work?Solved7.5KViews0likes6CommentsMirroring of postgres data in fabric
Hi, I want to use data from my postgres database in fabric. Fabric offers a way to mirror data but currently it is not available for postgres database. I have another way to use this data by loading it into a warehouse inside fabric, but this means I will be replicating my data. Is there any other way that would allow me to use this data in fabric without replicating it.Solved4.3KViews0likes7CommentsHow do I organize everything?
Hi all, My organisation is on the verge of implementing Fabric, we think it's a great total solution for BI & Analytics. My task is to set up everything, from ADF pipelines to BI reports. Until now we have only used Power BI for our reporting needs. Everything else within Fabric is completely new to us. Here's the situation, like most businesses we are dealing with operational, management, financial and HR data. Currently I have those reports set up in different workspaces, with different people having access to them. Currently we are querying directly on our operational databases. Of course we want to change that and set up our data in the Fabric Cloud. When organizing all of this, what are some best practices regarding data governance in this situation? 1) Should I just set up 1 lakehouse with all the data for the whole tenant, and restrict access to tables/files? 2) Or should I make several data lakehouses (one for each data 'category')? 3) Or should I create several domains with one data lakehouse in each domain? Any insight would be much appreciated.5.5KViews4likes7CommentsHow to set default Lakehouse in the notebook programmatically?
Hi All, Here are my workspace setup: Workspace Fabric's Artifacts Git branch folder Fabric-Sandbox notebook: landingtoBronze lakehouse: Bronze feature/bronze123 Fabric-Dev notebook: landingtoBronze lakehouse: Bronze Dev Problem: Each time i updated the notebooks code to the Fabric-Dev workspace , i need to attach bronze lakehouse to "landingtoBronze" notebooks manually in the workspace so that it point to the right bronze lakehouse. Please note that each workspace has it own Bronze Lakehouse. For example, I was working in the "landingtoBronze" notebook in my feature branch in the Fabric-sandbox workspace. during the notebook development, I attached the bronze lakehouse of the workspace as default lakehouse to the notebook. once i had finished my notebook development, i created the pull request and merge the code from feature branch into dev branch. while getting the code updated in the Fabric-Dev workspace, I can see the default bronze database is actually referring to the Fabric Sandbox workspace. Is it possible to attach a default lakehouse in the notebook programmactically? Looking forward to hearing for any suggestion. it could be something that i am doing wrongly. Thanks.Solved25KViews0likes11CommentsLakehouse permissions not workign with Entra ID Groups
I have granted an Azure Entra ID group read access to the lakehouse/databases but the users within that group cannot login and see the data. When I grant the individual user read access they can login and query the tables. Reaching out to Anonymous to see if we can get some updates.Solved2.4KViews0likes7CommentsDirect Lake fallback to Direct Query due to SQL security?
Hi, I am wondering if Direct Lake mode will keep working if I apply granular SQL access control (GRANT, DENY, REVOKE) on a SQL Analytics Endpoint or Data Warehouse? I am also wondering if Direct Lake mode will keep working if I apply SQL Row Level Security on a SQL Analytics Endpoint or Data Warehouse? Thanks! 😀3.8KViews0likes6Comments