[Azure Data Factory] Data Flow Sink is extremely slow
I'm copying about 800 items from SAP to Azure Fabric. I'm running pipeline with data flow in Azure Data Factory (ADF). We are using Self-hosted integration runtime. The Data Flow is not complex, it just copies data and adds an additional column. There are no complex transformations. However, the Sink activity takes around 10 minutes! Sink configuration: Sink type: dataset Dataset: Fabric warehouse Allow schema drift: true Allow insert: true Batch size: 100 Enable staging: false (I cannot turn it on; otherwise, I get an error in my pipeline: "Fabric DW supports staging only via copy command - EXE-0001, Dataflow cannot be analyzed as a graph.") I also tried using Copy activity as suggested here, but it keeps failing with a BULK insert error: Failure happened on 'Sink' side. ErrorCode=SqlOperationFailed, 'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException, Message=A database operation failed. Please search error to get more details., Source=Microsoft.DataTransfer.Connectors.MSSQL, ''Type=Microsoft.Data.SqlClient.SqlException, Message=Incorrect syntax near 'INSERT BULK'., Source=Framework Microsoft SqlClient Data Provider,' Some other observations: Sink processing time: 10m 33s Stage time: 7m 36s Table operations SQL duration: 2m 30s I also checked in the warehouse's long_running_queries and saw that INSERT was called 840 times, with each insert taking around 450ms. That means 800 * 450ms = 360,000ms (6 minutes)! My Question: Why is ADF inserting items one by one and ignoring the batch size of 100? Since I cannot enable Staging without errors and BULK insert fails, what is the correct configuration to load data from SAP to Fabric efficiently?
Connection to BW4HANA from Power BI
Dear team, Im triying to connect to sap bw4hana using .net connector as mentioned in microsoft help. I tried installing the highlighted 3.0 connector with option install "install assemblies to GAC" , i was able to connect to sap bw app server. but when i use recent data sources for next time. it throws below message more over nagivation is very very slow. how can i find implementation version is 1.0 or 2.0 and in connection page implementation options is not coming can anyone share the exact connector and download media. Regards, Selva
Training Resources for Manufacturing and Quality
Can anyone recommend any YouTube or other training resources for power bi reporting with a focus on maufacturing, technical and quality issues? Seems most of the training out there is geared towards sales and sales trending. Okay some of the senarios can be adapted but it would be easier for me to garner information where it relates more to what I am doing. Currently trying to do complaint trending using two different raw data sources, SAP and SQL (although with the SQL data I am only allowed to connect via access databases!). So having to merge the two lots of production data and skus etc. But would like to see ideal example of star schemas used for this type of trending, to give me some pointers. Currently using a denormalised table scraped from a report in access, along with data provided via launch bi.
SAP Business Warehouse Message Server data source not supported on Power BI report server?
Hello everyone, I am facing an issue with connecting to an SAP Business Warehouse Message Server data source on Power BI report server, and I would appreciate some advice from the community on how to resolve it. When I try to connect to the SAP Business Warehouse Message Server data source on Power BI report server, I receive an error message stating that the data source is not supported. However, I am able to connect to the same data source without any issues using the Power BI desktop application. I have tried several troubleshooting steps, including ensuring that the necessary drivers are installed on the report server, but none of these steps have resolved the issue. I suspect that this could be a compatibility issue between SAP Business Warehouse Message Server and Power BI report server. However, I am not sure how to confirm this or what steps I can take to resolve it. Has anyone in the community encountered a similar issue, and if so, what steps did you take to resolve it? I would appreciate any advice or suggestions on how to resolve this issue. Thank you in advance for your help.
Convert SAP BO measure with Foreach() operator to DAX formula
Hi I'm pretty new to SAP BO and I'm trying to convert a measure that I've found in a report to DAX language. The foruma uses the context operator FOREACH() on two columns ([Data].[KPI Group] and Data].[KPI Name] ) and I don't know how to write it in DAX. Could you please help me figuring out? Here's the code: SUM( IF( [Not Available] FOREACH( [Data].[KPI Group];[Data].[KPI Name] )="Y" ) THEN 1 ELSE 0 ) Thanks
DAX subtotals P&L reporting Direct Query
Hello everyone, I am making a P&L in Power BI with a direct query connection with SAP data and a hierarchy from SAP. The situation is that the SAP hierarchy could not be changed. I want to format the hierarchy with some small changes. How do i fix this at the best way? Is it possible with DAX? In the attachment i have provided a p&L file and some example data. Revenue is a positive number and costs are negative. This two minus each other is the subtotal of the EBITDA. What i want to create is the following: GL_account_level 3 Revenue Gl_account_level 4 Revenue Revenue cars GL_account_level 3 Costs Gl_Account_level 4 Cost Loan costs EBITDA (subtotal) GL_account_level 3 Depreciation Gl_account_level 4 Depreciation Depreciation cars EBIT (subtotal) GL_account_level 3 Interest (Regular exploitationresult subtotal) GL_account_level_3 Extraordinary income and charge Subtotal result before tax The current situation is the following: EBIT (level 1) EBITDA (level 2) Revenue (level 3) Revenue cars (level 4) Costs (level 3) Loan costs (level 4) Interest and depreciation (level 2) Extraordinary income and charge (level 1) Extraordinary income and charge (level 2) Extraordinary income and charge (level 3) Total Power BI result before tax I am searching very long for a good way how to fix this, so if anyone can help me i'm very happy:)SAP BW 4 HANA - Power BI Report Server - No Connection
Hi all, trying to connect PBI Report Server to SAP BW 4 HANA (via SAP ODBC Driver). Data is shown in PBI Desktop (RS) but not when saved to the PBI Report Server (both on the same server located). Trying to check the data source (from Report Server) by using a database user gives the error: ERROR [IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified Anyone with the same problem / any advice to find a solution? Thanks in advance!
