fundamentals

89 Topics

"The remote certificate is invalid" error on pipeline Fabric job to copy parquet files from AWS S3
When I try to copy parquet files from AWS S3 bucket with "copy data" job on pipelline function, this error show up: "The file operation is failed. A WebException with status TrustFailure was thrown. The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. The remote certificate is invalid according to the validation procedure. Activity ID: ******" The test connection results OK. The shortcut function works well too. But we need to copy the data from S3 with pipeline function, the shortcut don't help us. We try with connection type "Amazon S3 Compatible" too and we get the same error. Some one can help or are getting this error too? Thanks!
soren
4 months ago Place Fabric platform
1.3KViews
0likes
2Comments
How do I organize everything?
Hi all, My organisation is on the verge of implementing Fabric, we think it's a great total solution for BI & Analytics. My task is to set up everything, from ADF pipelines to BI reports. Until now we have only used Power BI for our reporting needs. Everything else within Fabric is completely new to us. Here's the situation, like most businesses we are dealing with operational, management, financial and HR data. Currently I have those reports set up in different workspaces, with different people having access to them. Currently we are querying directly on our operational databases. Of course we want to change that and set up our data in the Fabric Cloud. When organizing all of this, what are some best practices regarding data governance in this situation? 1) Should I just set up 1 lakehouse with all the data for the whole tenant, and restrict access to tables/files? 2) Or should I make several data lakehouses (one for each data 'category')? 3) Or should I create several domains with one data lakehouse in each domain? Any insight would be much appreciated.
Noeleke1301
5 months ago Place Fabric platform
5.4KViews
4likes
7Comments
Naming Conventions For Fabric
Hi Team, Are there any standard naming conventions available in Fabric,If Yes can you please provide that one ?
chandukasani
7 months ago Place Fabric platform
17KViews
0likes
5Comments
How to generate Calendar table in warehouse
I need to generate a Calendar table - it is a very simple task for regular SQL but I'm using Microsoft Fabric and I've tried so far many options, however no success due to Fabric SQL limitations. My table is: CREATE TABLE dbo.CalendarData ( DateValue DATE, Year INT, StartOfYear VARCHAR(20), EndOfYear VARCHAR(20), Month INT, StartOfMonth VARCHAR(20), EndOfMonth VARCHAR(20), DaysInMonth INT, Day INT, DayName VARCHAR(20), DayOfWeek INT, MonthName VARCHAR(20), Quarter INT, StartOfQuarter VARCHAR(20), EndOfQuarter VARCHAR(20) ); Option 1: while loop says - Each SQL statement runs as an independent session. Session context does not persist across SQL statements. Learn more at https://learn.microsoft.com/en-us/fabric/data-warehouse/sql-query-editor#limitati DECLARE @StartDate DATE = '2023-01-01'; DECLARE @EndDate DATE = '2023-01-31'; WHILE @StartDate <= @EndDate BEGIN INSERT INTO dbo.CalendarData ( Date, Year, StartOfYear, EndOfYear, Month, MonthName, StartOfMonth, EndOfMonth, DaysInMonth, Day, DayName, DayOfWeek, Quarter, StartOfQuarter, EndOfQuarter ) VALUES ( @StartDate, YEAR(@StartDate), -- Year DATEFROMPARTS(YEAR(@StartDate), 1, 1), -- StartOfYear DATEFROMPARTS(YEAR(@StartDate), 12, 31), -- EndOfYear MONTH(@StartDate), -- Month DATEFROMPARTS(YEAR(@StartDate), MONTH(@StartDate), 1), -- StartOfMonth EOMONTH(@StartDate), -- EndOfMonth DAY(EOMONTH(@StartDate)), -- DaysInMonth DATENAME(MONTH, @StartDate), - MonthName DAY(@StartDate), -- Day DATENAME(WEEKDAY, @StartDate), -- DayName (DATEPART(weekday, @StartDate) + 5) % 7 + 1, -- DayOfWeek DATEPART(QUARTER, @StartDate), -- Quarter CAST(DATEADD(QUARTER, DATEDIFF(QUARTER, 0, @StartDate), 0) AS DATE), -- StartOfQuarter EOMONTH(DATEFROMPARTS(YEAR(@StartDate), DATEPART(QUARTER, @StartDate) * 3, 1)) -- EndOfQuarter ); SET @StartDate = DATEADD(day, 1, @StartDate); END; Option 2: CTE - The query references an object that is not supported in distributed processing mode WITH Numbers AS ( SELECT TOP (DATEDIFF(day, @startDate, @endDate) + 1) n = ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) FROM sys.all_columns ) INSERT INTO dbo.Calendar ( Date, Year, StartOfYear, EndOfYear, Month, MonthName, StartOfMonth, EndOfMonth, DaysInMonth, Day, DayName, DayOfWeek, Quarter, StartOfQuarter, EndOfQuarter ) SELECT DATEADD(day, n - 1, @startDate) AS Date , YEAR(DATEADD(day, n - 1, @startDate)) AS Year , DATEFROMPARTS(YEAR(DATEADD(day, n - 1, @startDate)), 1, 1) AS StartOfYear , DATEFROMPARTS(YEAR(DATEADD(day, n - 1, @startDate)), 12, 31) AS EndOfYear , MONTH(DATEADD(day, n - 1, @startDate)) AS Month , DATENAME(MONTH, DATEADD(day, n - 1, @startDate)) AS MonthName , DATEFROMPARTS(YEAR(DATEADD(day, n - 1, @startDate)), MONTH(DATEADD(day, n - 1, @startDate)), 1) AS StartOfMonth , EOMONTH(DATEADD(day, n - 1, @startDate)) AS EndOfMonth , DAY(EOMONTH(DATEADD(day, n - 1, @startDate))) AS DaysInMonth , DAY(DATEADD(day, n - 1, @startDate)) AS Day , DATENAME(WEEKDAY, DATEADD(day, n - 1, @startDate)) AS DayName , (DATEPART(weekday, DATEADD(day, n - 1, @startDate)) + 5) % 7 + 1 AS DayOfWeek , DATEPART(QUARTER, DATEADD(day, n - 1, @startDate)) AS Quarter , CAST(DATEADD(QUARTER, DATEDIFF(QUARTER, 0, DATEADD(day, n - 1, @startDate)), 0) AS DATE) AS StartOfQuarter , EOMONTH(DATEFROMPARTS(YEAR(DATEADD(day, n - 1, @startDate)), DATEPART(QUARTER, DATEADD(day, n - 1, @startDate)) * 3, 1)) EndOfQuarter FROM Numbers Option 3: Insert from Table function - The query references an object that is not supported in distributed processing mode. INSERT INTO dbo.Calendar ( Date, Year, StartOfYear, EndOfYear, Month, MonthName, StartOfMonth, EndOfMonth, DaysInMonth, Day, DayName, DayOfWeek, Quarter, StartOfQuarter, EndOfQuarter ) SELECT dr.Date AS Date , YEAR(dr.Date) AS Year , DATEFROMPARTS(YEAR(dr.Date), 1, 1) AS StartOfYear , DATEFROMPARTS(YEAR(dr.Date), 12, 31) AS EndOfYear , MONTH(dr.Date) AS Month , DATENAME(MONTH, dr.Date) AS MonthName , DATEFROMPARTS(YEAR(dr.Date), MONTH(dr.Date), 1) AS StartOfMonth , EOMONTH(dr.Date) AS EndOfMonth , DAY(EOMONTH(dr.Date)) AS DaysInMonth , DAY(dr.Date) AS Day , DATENAME(WEEKDAY, dr.Date) AS DayName , (DATEPART(weekday, dr.Date) + 5) % 7 + 1 AS DayOfWeek , DATEPART(QUARTER, dr.Date) AS Quarter , CAST(DATEADD(QUARTER, DATEDIFF(QUARTER, 0, dr.Date), 0) AS DATE) AS StartOfQuarter , EOMONTH(DATEFROMPARTS(YEAR(dr.Date), DATEPART(QUARTER, dr.Date) * 3, 1)) EndOfQuarter FROM dbo.GenerateDateRange(@startDate, @endDate) AS dr; How to generate simple data for table in Fabric??
Solved
Marusyk
1 year ago Place Fabric platform
11KViews
0likes
9Comments
unable to run multiple notebooks on different user sessions
Hello, The problem: We are a team of 4. Currently 2 of us are getting into Fabric but it is really frustrating. When user A is running a notebook, user B is not able to run notebooks and vice versa. What do we need to configure in order to have concurrent users running notebooks. Thanks and regards.
Solved
Anonymous
1 year ago Place Fabric platform
3.4KViews
0likes
9Comments
How to enable Microsoft Copilot in Microsoft fabrics !!
Can't able to find copilot in microsoft fabrics , also not able to find how to install it(if we have to install it or add in from somewhere). Please help Thankyou !!
Anonymous
1 year ago Place Fabric platform
1.6KViews
0likes
2Comments
External Tables in Fabric
Hello, reading and understanding more about shortcuts in Fabric it seems like Unmanaged / External Tables are useless. The main point of an external Table in Spark is to leave data where it is and have a table which references this data. Thus only metadata is managed by spark and deleting this table does not effect the original data. Is this not exactly what a shortcut in fabric is? Why would I even create an External Table in Fabric and what is the point of a shortcut external table, which is technically possible but does not seem to make much sense.
Solved
mpolonskiy5637
1 year ago Place Fabric platform
8.2KViews
0likes
2Comments
Is anyone else getting a Fabric data warehouse error stopping sprocs from showing/executing?
Hi all I've raised a support ticket but was just wondering if anyone else is getting the following error when trying to access any stored procedures or functions. I can see tables and views in our workspaces and in Azure Data Studio, but anything "programmability"-related doesn't show ... it doesn't appear to be able to execute any stored procedures either. TypeError: Cannot read properties of undefined (reading 'model') Cheers James
PhantomThread
1 year ago Place Fabric platform
4KViews
0likes
10Comments
Cannot see some Fabric tables from Power BI Desktop
I have loaded several tables into a Fabric Lakehouse. When I try to connect from Power BI Desktop to the lakehouse, I can only see one of the tables and the lakehouse logs, not all three of the tables that exist there. Is this a permissions issue? Or a versioning issue?
mtitsworthsfmfb
1 year ago Place Fabric platform
5.1KViews
1like
5Comments
How to truncate a lakehouse table.
I have a multiple dataflowgen2 inserts data into the same lakehouse table. Before inserting data into lakehouse table, I would like to truncate the lakehouse table. How can I do that. The lakehouse table already have 40 milions rows.
Solved
Dinesh09
1 year ago Place Data Engineering
11KViews
1like
9Comments