Red Flag in SQL Analytics EndPoint - User Identity Access Mode
We are seeing some red flags when switching the default access mode "Delegated" to ""User's Identity" in a SQL Analytics EndPoint of lakehouse, but seeing red flags with message not relevant to the user who were given access to downstream lakehouse, below is the details. Current Configuration: LH1 is in the hub workspace/WS1 with Onelake security role where users were added. In the user workspace/WS2, a shortcut to LH2 the source data is created, user can access through LH2’s SQL Analytics Endpoint, data access mode is switched to User’s Identity, so that source lakehouse's security role control access fully. In SQL End, there are some red flags as below, but user can access data without issue, which is confusing: error details: User principal is not supported. Issue: Error details is specifically for an AD group which is a workspace contributor role in the source workspace role (default reader of all tables in the source lakehouse), and it doesn't have access to the user's lakehouse. Question: Does anyone see the same? or it's a known issue?
Fabric Welcome Popup - Hide/Disable
Team - I am back with another requirement from my end users. They dont want to see the "Welcome to Fabric View " pop-up when they log in using Fabric apps URL. I have gone through all the tenant setting and I dont see any way to hide or disable this pop-up. Did anyone come across similar requirement? Any thoughts or suggestions? -PattHigh CU consumption after enabling Workspace Monitoring
Hi everyone, I’m wondering if anyone else has experienced very high CU consumption after enabling Workspace Monitoring in Microsoft Fabric. In my case, I tested it on an F4 capacity, and the additional CU usage was significant enough that the capacity became difficult to use for the actual workloads running in the workspace. I understand that Workspace Monitoring creates and uses Fabric resources for collecting and storing logs, so some additional capacity consumption is expected. However, the overhead I’m seeing feels quite high and it makes no sense to enable this feature I’m interested if there are any answers on following questions: Have you noticed a significant CU increase after enabling Workspace Monitoring? Is the impact especially noticeable on smaller capacities such as F2/F4/F8? Have you found any practical way to reduce the monitoring overhead? Are you using Workspace Monitoring in production on smaller capacities, or do you avoid it because of the CU cost? At the moment, my concern is that on smaller capacities the monitoring workload itself can consume enough capacity to materially affect the workloads it is supposed to help monitor. Would be interested to hear about other people’s experience and whether this is expected behaviour.
How to efficiently maintain complex multi-table aggregations incrementally in Microsoft Fabric?
How to efficiently maintain complex multi-table aggregations incrementally in Microsoft Fabric? Comparison with Snowflake Dynamic Tables Hi everyone, I have a scenario where source data is continuously/incrementally loaded into Microsoft Fabric. We have multiple entities such as Customer, Product, Orders, Sales, Inventory, Invoices and Payments. We need to create business-ready/precomputed tables by joining multiple entities and performing aggregations. For example, a simplified transformation could be Customer + Orders + Product, followed by joins and aggregation by Customer, Product and Month. The challenge is not simply adding new records. A new incremental record can change an existing aggregation group. For example, suppose we already have Customer C001, Product P001, Month Jan and Sales of 1,000. If a new order arrives for the same Customer, Product and Month with Sales of 250, the existing aggregate needs to become 1,250. This becomes more challenging when the transformation involves multiple joins, GROUP BY, aggregations, DISTINCT counts, multiple related entities, and updates or deletes in source data. If we rebuild the complete precomputed table during every refresh, a large amount of historical data may need to be recomputed. As data volume and the number of entities increase, this could result in more data, larger transformations, more compute and longer refresh times. We are therefore trying to determine the best incremental maintenance pattern for these types of tables in Microsoft Fabric. SNOWFLAKE COMPARISON One approach we have been looking at in Snowflake is Dynamic Tables. Dynamic Tables provide a declarative way to define transformed or materialized results and allow Snowflake to manage refreshes based on changes and the configured refresh mode and target lag. The capability we are particularly interested in is maintaining the derived result incrementally instead of treating every refresh as a complete rebuild. We also understand that Snowflake Dynamic Tables do not make every transformation automatically incremental. Query shape, joins, aggregations and unsupported constructs can affect whether incremental refresh is possible. FABRIC OPTIONS WE ARE EVALUATING The first option we are evaluating is Lakehouse Materialized Lake Views (MLVs). MLVs appear to provide a similar architectural pattern where source tables are transformed into a persisted materialized result and the platform manages refresh. We are particularly interested in the optimal refresh and incremental refresh capabilities. We would like to understand how well MLVs handle transformations involving multiple joins, GROUP BY, aggregations, large historical datasets and new records that affect existing aggregation groups. For example, if Customer, Orders and Product are joined and aggregated by Customer, Product and Month, and a new order affects an existing historical Customer + Product + Month group, how does MLV incremental refresh handle this? The second option we are considering is maintaining physical precomputed or serving tables in Fabric Warehouse. The architecture would be roughly Lakehouse, Clean and Validated Data, Transformation and Aggregation, Warehouse Precomputed or Serving Table, Semantic Model and Power BI. One possibility is to use CTAS or staging-based patterns to generate the serving tables. However, if the transformation is rebuilt from the complete historical dataset, we may still end up recomputing a large amount of data during every refresh. We would therefore like to understand whether there is a recommended Fabric pattern for maintaining these physical tables incrementally, particularly when new or changed source records can affect existing aggregate groups. WHAT WE ARE TRYING TO DETERMINE We are not trying to claim that one approach is better than another. We are trying to identify the closest Fabric architecture to the incremental-maintenance capability we are familiar with from Snowflake Dynamic Tables. Can MLVs efficiently maintain complex multi-table joins and aggregations incrementally as data grows? What types of SQL transformations cause MLVs to fall back to full refresh? How does MLV incremental refresh behave when new records modify an existing aggregation group? For complex transformations, is it better to break the logic into multiple MLVs or intermediate layers? If using Fabric Warehouse physical serving tables, what is the recommended pattern for avoiding full historical recomputation? Are there Fabric-native patterns for identifying and recomputing only the affected partitions, keys or aggregation groups? For large enterprise datasets, what approach have others found most scalable and maintainable? OUR CURRENT ARCHITECTURE Our architecture is roughly Source Systems, Bronze, Clean and Validated Data, Precomputed or Serving Tables, Gold or Consumption Layer, Semantic Model and Power BI. The objective is to perform expensive joins and aggregations during data processing rather than repeatedly during interactive BI queries. The open question is how best to maintain the Precomputed or Serving layer incrementally as source data continues to grow and change. I would really appreciate input from anyone who has implemented this at scale in Microsoft Fabric, particularly comparisons between Fabric MLVs, Warehouse-based serving tables and other Fabric-native incremental processing patterns. Thanks in advance!Fabric IQ: Do Ontology entity synonyms work with Data Agent?
Hi, everyone! Short intro Currently I'm using the "super-duper-mega-nano-ultra" product - Microsoft Fabric to build natural language processing flow on-top of Microsoft Fabric Warehouse data. As for now It's rather a POC than production solution. I found a lot of the official Microsoft' documentation related to my task, but I have a little problem... The solution architecture (high-level) I made some investigations and as the result is the following architecture, which I want to implement (picture below): The idea is the next: the Microsoft Fabric Warehouse schema is connected to Microsoft Fabric Lakehouse, using shortcut; the Microsoft Fabric Ontology consumes the Microsoft Fabric Lakehouse as a data source for data binding; the Microsoft Fabric Data Agent uses a Microsoft Fabric Ontology (enriched with business context) to process natural language questions. The Microsoft Fabric Warehouse contains the following (dummy) objects (picture below): Tenant settings Microsoft documentation says, that specific tenant configurations should be applied to use Microsoft Fabric Ontology with Microsoft Fabric Data Agent (https://learn.microsoft.com/en-us/fabric/data-science/data-agent-tenant-settings) - everything is configured properly. Ontology configuration My Microsoft Fabric Ontology is configured as below (the configuration is influenced by Microsoft Fabric Ontology tutorial, which can be found here - https://learn.microsoft.com/en-us/fabric/iq/ontology/overview). Main view - two entities with a single relationship: ETLEntity entity configuration - the entity has the description, one synonym, metadata: ETLEntityRun entity configuration - the entity has the description, one synonym, metadata: As for now the Microsoft documentation says, that Microsoft Fabric Ontology descriptions, synonyms, metadata help Microsoft Fabric Data Agent to better understand the context (https://learn.microsoft.com/en-us/fabric/iq/ontology/how-to-add-semantic-enrichment). The problem My Microsoft Fabric Data Agent is connected to my Microsoft Fabric Ontology, which is described above, but the agent can't answer the simple questions about entities and the questions examples are provided below (the Microsoft Fabric Ontology Graph model was refreshed successfully before questions were asked): ETLEntity successful question without synonym usage: ETLEntityRun successful question without synonym usage: ETLEntity failed question with synonym usage: ETLEntityRun failed question with synonym usage: Looks like Microsoft Fabric Data Agent can't figure out, which entities are unicorn/wizard, even if they have appropriate synonyms. It's not my first iteration - I tried a lot, but result still the same every time. I feel like I missed something obvious in my configuration, but what... What are your thoughts? P.S.: the provided configuration is simple; objects and their metadata has no business context - It's just a sample, which I built to test some scenario; I think It's enough to check such use-case.
Users that subscribe to PowerBi Paginated Report from another tenant
Dear all, One of our project teams developed a custom Fabric notebook that uses the Graph REST API to send a paginated report to multiple users (mailboxes). This custom notebook extracts the report, saves it as an Excel file, and sends it to multiple mailboxes outside the tenant. From what I understand, this approach was taken because, as the users belong to a different tenant, they would not be able to receive the email or subscribe to the report directly. However, I have a feeling there may be a workaround for this. How can users subscribe to a paginated report and receive it via email if they are not part of the tenant? Would it be sufficient to invite these users into our Entra ID as guests — would they then be able to subscribe to the report (themselfs) and receive it in their external mailboxes? Or is the Power BI Service only able to send emails to mailboxes within the same tenant? If the Power BI Service does allow guests to subscribe and receive emails, do we also need to unblock any other settings on the Fabric Admin page, alongside adding them as guests? Many thanks, Pedro
