data engineering
9 TopicsSend Email using Python /PySpark Notebook
Hi Team, I would like to send email when my Notebook fails with any errors, which I am appending into "error_data" list; I want to do in Notebook and schedule that Notebook, so that if any error within Notebook execution, I will get an email Notification; I want to just do with Notebooks (Pipelines won't be required); and won't be needing any sender email; I just need to give receiver email. Any assistance on this will be highly helpful. Thanks in advance, AshwathSolved1.6KViews0likes4Commentscombinatorics in dinamic dax
Hello guys, i have posted here so many times about getting a logic right, but i just noticed it was impossible, in the way i was thinking, no one could help me. I have a new aproach of how to get the desired result, its by using combinatorics, of course, i need to know if its even possible in the dax. I have the image bellow that shows how the combinations might work, and ahead i also have the code i'm trying, its not working properly and i would like some help with it. In this code, the objective is have at the end the combinations that attend to the condition that the amount of concatenated orders is the highest and the sum of values are <= 21. Obs. this code might work with many different products, but to this example i just used one. Code: FinalCorrectStatus = VAR CurrentItem = SELECTEDVALUE('DB_ORDERS'[Produto]) VAR OrdersTable = FILTER(ALL('DB_ORDERS'), 'DB_ORDERS'[Produto] = CurrentItem) VAR OrderCombinations = ADDCOLUMNS( GENERATEALL( OrdersTable, SUMMARIZE( OrdersTable, 'DB_ORDERS'[Pedido], "Soma_Quant_Falta", SUM('DB_ORDERS'[QT. falt]) ) ), "PedidosConcatenados", CONCATENATEX(OrdersTable, 'DB_ORDERS'[Pedido], ", "), "Qtd_Pedidos", COUNTROWS(OrdersTable) -- Conta o número de pedidos na combinação ) VAR FilteredCombinations = FILTER(OrderCombinations, [Soma_Quant_Falta] <= 21) VAR MaxPedidos = MAXX(FilteredCombinations, [Qtd_Pedidos]) -- Encontra o maior número de pedidos VAR BestCombination = FILTER(FilteredCombinations, [Qtd_Pedidos] = MaxPedidos) -- Mantém apenas a combinação com mais pedidos RETURN BestCombination Can anyone please help me with it?Solved1.3KViews0likes5Commentssemantic model roles
1- the power bi desktop is connected to the microsoft fabric semantic model 1- I then updated the semantic model by adding roles 2- in the semantic model, RLS, I added users to the roles 4- back in the power bi desktop, when I click on view as roles, I do not see the roles or users added to the roles. Anything I am doing wrong? Thank you526Views0likes2CommentsImport the result of Postman queries to Power BI
Hi everyone, thank you for your help. I'm new to microsoft fabric and I find it magnificent. However, I am faced with a complex situation. I want to connect power bi to a database to extract the information. The API has been developed and offers two-factor authentication (Token and basic authentication). With postman I can visualize the data that is in my data box. when I try to connect the API to power BI I'm asked for authentication. Since I use two authentication systems, I'm a bit lost. Should I ask the depreper to set a single authentication system or should I run my requests in Postman and save each result in the JSON files before importing them into Power BI? Also I would like to know if it is possible to automate this process using the power automate platform?Solved1.9KViews0likes2CommentsUploading images into Lakehouse: How to use them in Power BI
Hi Community, I need your help regarding images in MS Fabric. It was told that we can upload images, pdf, etc., into MS Fabric through Lakehouse. Well, I tried and I succefully saved one image in the Lakehouse files. I cannot upload it to "Tables", I think for obvious reasons. But I was not able to find a way to use this image in a Power BI report. When we create a report from the Lakehouse SQL endpoint or warehouse, we can only see the tables, not the files. I also tried to copy the file path, it did not worked. So, could you please give me a hint or a path so I can start using the images stored in my Lakehouse or Warehouse in a Power BI report? Thank you! Kind regards!2.4KViews2likes2CommentsBlob not found | IO Error
Hi Everyone, Yesterday, I added Data to my Power BI visual from DATA WAREHOUSE. Today, It is giving me a 404 Error. Blob not found. Also, Other Visuals are properly showing data from the Same Warehouse. My guess is there is something in Measure I'm using on this table. Total Employees = CALCULATE(COUNTX(FILTER(Emp_Personal_Data,Emp_Personal_Data[tm_dateofjoining]<=max(HRMS_DateTable[Date]) && (ISBLANK(Emp_Personal_Data[tm_dateofrelease]) || Emp_Personal_Data[tm_dateofrelease] > max(HRMS_DateTable[Date]))),(Emp_Personal_Data[Full Name])),CROSSFILTER(Emp_Personal_Data[tm_dateofjoining],HRMS_DateTable[Date],None)) This is the measure I'm using. Please help me out here. THis is happening again and again.746Views0likes1CommentLakehouse: Slow data transfer speeds when loading data into Power BI
Hi all! I've got this Lakehouse that I'd like to use as a source for a PBI report, but I run into slow data transfer speeds that prohibit data refresh, both in Desktop and in the service. My report use data from MS Defender. The Lakehouse consist of daily snapshots of approx. 3.5 million rows, distributed over 5-6 tables. I've currently got data for 40+ days. Loading data from the Lakehouse to Desktop for a single day (3.5 million rows) takes 1270s. As a comparison, loading the same amount of data from the Defender API into Desktop takes 35s. The refresh in the service runs for exactly one hour before failing when the connection is closed, likely by Fabric. I've enabled incremental reload for the dataset, but I need to perform a full fresh after I've published the dataset; it's infeasible to load all data into Desktop, as this would require 10-12h, using the load time for a single day as benchmark. I’ve made the refresh as parallel as possible by increasing the relevant settings in Desktop, and some of those should be carried over to the service. Any ideas about the cause the slow refresh speeds? Error message during refresh: [ValidateMarkupTags][ccon]DataSource.Error: An error happened while reading data from the provider: '[ccon]Invalid operation. The connection is closed.[/ccon]'. DataSourceKind = [ccon]SQL[/ccon]. DataSourcePath = [ccon]xxxx.datawarehouse.pbidedicated.windows.net;LH_LH_CUR[/ccon]. [/ccon]. The exception was raised by the IDataReader interface. Please review the error message and provider documentation for further information and corrective action. Thanks!1.8KViews0likes6CommentsCopilot is not opening in Power Bi Service
Hi Team , Getting below message while opening Coilot , earlier it was opening,Solved1.6KViews0likes3CommentsLakehouse: give different access to files/tables to different users
Hi to all, not sure if this is the right place. I'd like to understand if it's possible or is in road-map to give different access to files or tables available in a Lakehouse to different users. At the moment from what I can read here it's not possible, but seems a bit strong: https://learn.microsoft.com/en-us/fabric/data-engineering/workspace-roles-lakehouse So each users can see every table and file in a workspace. And if a user is more than a viewer has all CRUD operations on each element. How can we segregate? Only by creating more workspaces/Lakehouses? Best regards Daniele Tiles383Views0likes0Comments