dataflow
4345 TopicsFabric Data Agent Access issue
I have created a fabric data agent using semantic model of a published report, It is working fine but I'm not able to share it with others I published the agent in M365 copilot and share with my teams of 100 people only 4 or 5 people can go to M365 copilot and search the agent and added the agent to their agent tab others are not even seeing the agent but if I share the agent link from M365 copilot then they see the agent but agent is not answering the questions but it's perfectly answering for me and other 4 to 5 people who see the agent when I surfed it said it said there are 2 level of access to get answer from the agent 1. agent level access 2. underlying data level access I have workspace contributor access for workspace, read, write, reshare permission for the agent. most of them share the same access but there are not able to see the only difference I found is I'm having premium per user license and most of other have pro license so I doubted it but internet says license is not an issue can anyone pls help me with this issue59Views0likes10CommentsF64 Import Model Refresh Fails Due to Memory: Can Data Duplication Be Avoided
Hello, We are experiencing refresh failures for a large Power BI Import semantic model on an F64 Microsoft Fabric capacity. The refresh fails because the operation exceeds the available memory. Our main objective is to resolve the refresh failure. We are also trying to understand how the model is compressed and which objects consume the most storage, because reducing the model size may help reduce the refresh-time memory requirement. Current Environment The semantic model contains three fact tables and multiple dimension tables. The data source is an Azure Synapse Analytics dedicated SQL pool. The semantic model and reports run on an F64 Microsoft Fabric capacity. The semantic model uses Import storage mode. The semantic model size is approximately 14.8 GB. We import only the columns required for reporting, relationships, calculations, and security. One fact table contains approximately 3.8 billion rows. This large fact table is divided into partitions of approximately 100 million rows using XMLA. We refresh only the partitions that contain changed data. The other two fact tables have been denormalized by adding only the required descriptive attributes that were previously obtained from dimension tables. We denormalized those two fact tables because some table visuals were failing with the following error when using the normalized design: Query exceeded available resources. The denormalized design improved the table visual behaviour, but it increased the semantic model size. Main issue: Refresh failure due to memory usage Even though we refresh only the affected partitions of the 3.8-billion-row fact table, the semantic model refresh is failing because of the memory constraint on the F64 capacity. We would like to understand how memory is used during an Import semantic model refresh. Our understanding is that Power BI needs to preserve the currently queryable version of the model or partition while it processes a new version. This could temporarily require memory for both existing and newly processed data, along with additional memory for compression, dictionaries, relationships, indexes, and transaction commit operations. Could you please clarify the exact refresh behaviour? Specifically: Does Power BI temporarily maintain two copies of the complete semantic model, or only two versions of the partition being refreshed? Which model structures may be duplicated or rebuilt during a partition refresh? How much additional working memory is normally required beyond the stored semantic model size? Could a 14.8 GB semantic model exceed the F64 memory limit during refresh because the existing model, refreshed partition, dictionaries, relationship indexes, and processing workspace are held in memory at the same time? Does refreshing one partition at a time materially reduce peak memory, or can model-level structures still cause high memory consumption? Can refresh-time model duplication be disabled? Is there any supported way to prevent or reduce the apparent duplication of data during refresh? For example, can an Import refresh directly modify or update the existing data in a partition instead of building a separate replacement version and switching to it after processing completes? We would like to know whether any of the following is possible: Update existing compressed data in place. Append new rows directly to an existing processed partition. Delete or modify individual rows without recreating the affected partition. Disable the refresh transaction or shadow-copy behaviour. Disable the retention of the old partition version during processing. Commit the refresh in smaller stages to reduce peak memory. Release the old partition from memory before loading the replacement. Process only model metadata, relationships, or calculations without creating another data copy. Configure a lower-memory refresh mode for large Import semantic models. Control the number of rows or segments processed in each refresh transaction. If in-place updates or disabling refresh-time duplication are not supported, what is the recommended approach for refreshing a model of this size on F64? We understand that transactional refresh behaviour may be required to keep the existing semantic model available to report users and to allow rollback if processing fails. However, we would like confirmation of whether this behaviour can be changed or optimized. VertiPaq dictionary and compression question We are also investigating which columns contribute most to the semantic model size. Consider the following simplified example: LargeFact[EntityIdentifier] SecondFact[EntityIdentifier] EntityDimension[EntityIdentifier] All three columns use the same data type and contain many of the same identifier values. Does VertiPaq create: one shared dictionary for the identifier across the complete semantic model, one dictionary per table, one independent dictionary per physical column, or separate dictionaries or encoding structures at the partition or segment level? Does the relationship between the fact and dimension tables allow the key dictionaries to be reused, or does every physical column maintain its own dictionary and encoded value storage? For example, suppose the fact table containing 3.8 billion rows has a foreign-key column referencing another table. If that foreign-key column contains substantially fewer distinct values than the total row count, is its storage broadly composed of: a dictionary containing the distinct values, encoded references for the 3.8 billion rows, column segments, relationship indexes, hierarchy structures, and other internal storage objects? We would like to understand whether the storage is primarily caused by: the dictionary, the encoded values across 3.8 billion rows, internal relationship structures, partition-related structures, or a combination of these. Storage query used I used the following DAX query to identify the columns and internal storage objects consuming the most space: // Check largest storage entity EVALUATE VAR SegmentSizes = GROUPBY ( INFO.STORAGETABLECOLUMNSEGMENTS(), [TABLE_ID], [COLUMN_ID], "Records", SUMX ( CURRENTGROUP(), [RECORDS_COUNT] ), "SegmentUsedBytes", SUMX ( CURRENTGROUP(), [USED_SIZE] ), "SegmentAllocatedBytes", SUMX ( CURRENTGROUP(), [ALLOCATED_SIZE] ) ) VAR ColumnDetails = SELECTCOLUMNS ( INFO.STORAGETABLECOLUMNS(), "TABLE_ID", [TABLE_ID], "COLUMN_ID", [COLUMN_ID], "Table", [DIMENSION_NAME], "Column", [ATTRIBUTE_NAME], "ColumnType", [COLUMN_TYPE], "DataType", [DATATYPE], "Encoding", [COLUMN_ENCODING], "DictionaryBytes", [DICTIONARY_SIZE] ) VAR Combined = NATURALLEFTOUTERJOIN ( SegmentSizes, ColumnDetails ) RETURN SELECTCOLUMNS ( Combined, "Table", [Table], "Column or object", [Column], "Object type", [ColumnType], "Data type", [DataType], "Encoding", [Encoding], "Records", [Records], "Segment MB", DIVIDE ( [SegmentUsedBytes], 1024 * 1024 ), "Dictionary MB", DIVIDE ( [DictionaryBytes], 1024 * 1024 ), "Approximate total MB", DIVIDE ( [SegmentUsedBytes] + COALESCE ( [DictionaryBytes], 0 ), 1024 * 1024 ), "Allocated MB", DIVIDE ( [SegmentAllocatedBytes], 1024 * 1024 ) ) ORDER BY [Approximate total MB] DESC The query showed identifier columns from multiple tables among the largest storage objects. The most significant result was a foreign-key column in the 3.8-billion-row fact table. The query reported that this column, or its related internal storage object, was consuming up to approximately 13 GB. Could you please confirm whether this query correctly estimates storage usage by column or internal storage object? Main questions Our primary questions are: How can we resolve the refresh failure caused by the F64 memory constraint? Does Import refresh temporarily duplicate the complete model, or only the affected partition and related structures? Is there any supported way to update an existing Import partition in place? Can refresh-time duplication or transactional processing be disabled? Is there a supported configuration that reduces peak refresh memory? Is the storage query above correct, particularly the approximately 13 GB reported for a foreign-key object? Are VertiPaq dictionaries maintained per column, per table, per partition, or per model? What architecture would Microsoft recommend for an Import semantic model containing a 3.8-billion-row fact table on F64? Our immediate objective is to complete the refresh successfully. Model-size optimization is important mainly because it may reduce the peak memory required during processing. At the same time, we need to avoid reintroducing the 'Query exceeded available resources.' error in the report table visuals. Thank you.59Views2likes3CommentsOne of my Dataflow Gen 1 stopped working
Hello everyone, I have a few DataFlow Gen1, all of them are working except one. "TauxHorairePanierMoyen" worked perfectly for months, and this morning I can't update the file. It shows this kind of error : Error: Request ID: 82fd0734-bf12-4ff2-b85d-79e2981cb408 Activity ID: 0b7f17c0-b8c4-4f96-beef-896adb689b04 When I go to Power Query it loads perfectly, but when I update, it doesn't work. --> It's a combined Excel Files. Requested on Nom du flux de données État d'actualisation du flux de données Nom de la table Nom de partition Actualiser l’état Heure de début Heure de fin Durée Lignes traitées Octets traités (Ko) Validation maximale (Ko) Temps processeur Temps d’attente Moteur de calcul Erreur 30/09/2026 11:29 TauxHorairePanierMoyen Échec TauxHorairePanierMoyen Non disponible Échec 30/09/2026 11:29 30/09/2026 11:35 00:06:00.7300 Non disponible Non disponible Non disponible Non disponible Non disponible Non disponible Error: Request ID: 82fd0734-bf12-4ff2-b85d-79e2981cb408 Activity ID: 0b7f17c0-b8c4-4f96-beef-896adb689b04 My Power Query code : Source = SharePoint.Files("XXX", [ApiVersion = 15]), #"Lignes filtrées" = Table.SelectRows(Source, each Text.Contains([Name], "TauxHorairePanierMoyen")), #"Fichiers masqués filtrés" = Table.SelectRows(#"Lignes filtrées", each [Attributes]?[Hidden]? <> true), #"Appeler une fonction personnalisée" = Table.AddColumn(#"Fichiers masqués filtrés", "Transformer le fichier", each #"Transformer le fichier"([Content])), #"Colonnes renommées" = Table.RenameColumns(#"Appeler une fonction personnalisée", {{"Name", "Source.Name"}}), #"Autres colonnes supprimées" = Table.SelectColumns(#"Colonnes renommées", {"Source.Name", "Transformer le fichier"}), #"Colonne de table développée" = Table.ExpandTableColumn(#"Autres colonnes supprimées", "Transformer le fichier", Table.ColumnNames(#"Transformer le fichier"(#"Exemple de fichier"))), #"Type de colonne changé" = Table.TransformColumnTypes(#"Colonne de table développée", {{"Nom Affaire", type text}, {"ORV", type text}, {"Nature ORV", type text}, {"Code équipe", type text}, {"Libellé équipe", type text}, {"N° client", type text}, {"Nom client", type text}, {"Nature (PGC)", type text}, {"Type ligne (PR/MO/DIV)", type text}, {"Date facture", type date}, {"No facture", Int64.Type}, {"Libellé réceptionnaire", type text}, {"Référence", type text}, {"Désignation", type text}, {"Quantité servie", type number}, {"Montant net", type number}, {"Montant brut", type number}, {"Montant remise", type number}, {"Montant remise CBQ", Int64.Type}, {"Remise client (Code)", type text}, {"Libellé remise CBQ", type text}, {"Famille comptable Reconstitué (Code)", type text}}) in #"Type de colonne changé" File Exemple : let Source = SharePoint.Files("XXX", [ApiVersion = 15]), #"Lignes filtrées" = Table.SelectRows(Source, each Text.Contains([Name], "TauxHorairePanierMoyen")), #"Fichiers masqués filtrés" = Table.SelectRows(#"Lignes filtrées", each [Attributes]?[Hidden]? <> true), Navigation = #"Fichiers masqués filtrés"{0}[Content] in Navigation Parameter let Source = Excel.Workbook(Paramètre, null, true), Navigation = Source{[Item = "TauxHoraire&PanierMoyen", Kind = "Sheet"]}[Data], #"En-têtes promus" = Table.PromoteHeaders(Navigation, [PromoteAllScalars = true]) in #"En-têtes promus" Transform File Exemple : let Paramètre = #"Exemple de fichier" meta [IsParameterQuery = true, IsParameterQueryRequired = false, Type = type binary, BinaryIdentifier = #"Exemple de fichier"] in Paramètre Thanks for your help. Kinds regards, Aude-Marie53Views0likes2CommentsPower BI Report Usage Metrics – Workspace-Level and Organization-Level Usage
Power BI Report Usage Metrics – Workspace-Level and Organization-Level Usage Hi everyone, I am working on a Power BI governance requirement around report usage metrics, and I am trying to understand the recommended approach at both the workspace level and organization/tenant level. I have some initial understanding of the available options, but I would like to get confirmation from the community and understand if there are better/native approaches. Workspace-Level Report Usage If I have multiple reports within a single workspace, I would like to see the usage metrics for ALL reports in that workspace in one place. For example: Workspace A Report 1 → Views / Unique Viewers / Last Viewed Report 2 → Views / Unique Viewers / Last Viewed Report 3 → Views / Unique Viewers / Last Viewed Instead of opening Usage Metrics separately for each report. My understanding is that the Usage Metrics semantic model can potentially be used to create a custom usage report for the workspace. However, I would like to confirm: How exactly can we get usage metrics for all reports within a single workspace? Is there a supported way to remove/modify the report-level filtering so that the Usage Metrics semantic model covers all reports in that workspace? Is this the recommended approach for workspace-level usage reporting? Are there any limitations with this approach? Organization-Level Report Usage The next requirement is to go beyond a single workspace. I would like to build a centralized view such as: Organization → Workspace → Report → Views → Unique Viewers → Last Viewed → Other Usage Metrics For example: Organization Workspace A Report 1 Report 2 Report 3 Workspace B Report 4 Report 5 Workspace C Report 6 Report 7 All of this should ideally be available in one centralized Power BI governance report. For this requirement, I know there are some possible approaches, such as: Using Power BI/Fabric Admin Portal capabilities with the required admin access Using Power BI Admin APIs Using Power BI Activity Log APIs Building a custom data collection process and storing the information in a Fabric Lakehouse/Warehouse/SQL database But my questions are: Are these the only practical/supported ways to achieve organization-wide report usage? Is there any native Power BI/Fabric feature that already provides tenant-level usage metrics across all workspaces? Can the standard Usage Metrics semantic model be consolidated across multiple workspaces? Is there another Microsoft-supported API or Fabric capability that is better suited for this requirement? What permissions/admin roles are actually required for the different approaches? Workspace-Level vs Organization-Level I am particularly interested in understanding whether the recommended architecture is different for these two scenarios: Scenario A: Single Workspace → All Reports → Usage Metrics Scenario B: Entire Organization → All Workspaces → All Reports → Usage Metrics Would we use the Usage Metrics semantic model for Scenario A and Admin APIs/Activity Logs or another centralized source for Scenario B? Or is there a common approach that can handle both? Historical Usage Another concern is data retention. My understanding is that the current Usage Metrics experience has a limited historical window, with the new Usage Metrics experience retaining 30 days of data. If we need to retain usage information for: 6 months 1 year Multiple years what is the recommended approach? Would this be the appropriate architecture? Power BI / Fabric Usage Data → API / Activity Log / Other Source → Fabric Lakehouse / Warehouse → Historical Usage Table → Centralized Governance Dashboard Or is there a Microsoft-native approach that can retain this historical information without building our own storage layer? Final Architecture Question Ultimately, I am trying to determine the recommended approach for: Organization ↓ Workspace ↓ Report ↓ Usage Metrics ↓ Historical Usage I would appreciate guidance on: How to achieve workspace-level usage for all reports How to achieve organization-wide usage across all workspaces Whether Admin Portal/API/Activity Log approaches are the only options Whether there is a native Fabric/Power BI capability I may be missing How to handle the 30-day Usage Metrics retention limitation Recommended architecture for long-term Power BI governance and usage tracking Thanks in advance to anyone who can share their experience or recommended approach.95Views1like4CommentsCheck the incremental refresh
Hello, How can I verify that the incremental refresh has been successfully applied after a dataset refresh? Are there any logs, indicators, or recommended methods to confirm that the refresh was processed incrementally rather than as a full refresh? Thank you.Solved59Views0likes3CommentsBackward Deployment
Hello ! I have a deployment pipeline in this order : A (Development) -> B (Preproduction) -> C (Production). This is the standard order of the workspace, but there were some reports that have been created directly in B, so is it possible to deploy from B to A? without unassigning the worskpace from the actual pipeline. Thank you.Solved119Views0likes6CommentsDataflow Gen1 officially marked as Legacy today — Pro users left with no migration path unless they
Microsoft just published a blog post today (April 2, 2026) titled “Dataflows: Thank you for eight years of Gen1—and why Gen2 is the future” by Miguel Llopis. Here’s the TL;DR: ∙ Dataflow Gen1 is now officially Legacy. No new features, limited support going forward. ∙ Retirement dates are “being finalized.” Premium customers will get 12 months’ notice. ∙ For Pro and PPU users? No timeline commitment — just a “strong recommendation” to move to Gen2. ∙ All future investment goes to Gen2 only. Here’s the problem: Dataflow Gen2 requires a Fabric capacity. It does not work on Pro workspaces. Never has. So if you’re a small team or a solo analyst running Gen1 dataflows on a Pro license — which is exactly the self-service scenario Microsoft marketed for years — you’re now being told: 1. The tool you rely on is dead. 2. The replacement requires a completely different (and more expensive) licensing tier. 3. There is no Gen2-on-Pro option. Not even a lightweight version. This is not a migration path. This is an upsell disguised as a deprecation notice. For context, Gen1 on Pro was already limited — no computed entities, no linked entities, no DirectQuery, no incremental refresh, no Enhanced Compute Engine. It was a basic but functional way to centralize Power Query transformations across multiple semantic models. Many of us built real pipelines around it. Now we’re told that even this basic capability has no future unless we start paying for Fabric capacity (minimum F2 at ~$260/month) on top of our existing Pro licenses. What I’d like to see from Microsoft: ∙ Either bring a limited Dataflow Gen2 experience to Pro workspaces (even without destinations, just matching current Gen1-on-Pro functionality) ∙ Or commit to a clear, long-term support timeline for Gen1 on Pro — not a vague “foreseeable future” ∙ At minimum, be transparent: if the plan is to force everyone onto Fabric, say it plainly instead of wrapping it in “we recommend you plan your upgrade” Anyone else impacted by this? Curious to hear how other Pro-license shops are thinking about it. Link to the blog post: https://powerbi.microsoft.com/en-us/blog/dataflows-thank-you-for-eight-years-of-gen1-and-why-gen2-is-the-future9.7KViews13likes15CommentsExrtract all workspace lineage list in service
Hi Experts, I need a help on how to extract the lineage of all workspaces, reports inside those workspaces and data models connected to that reports and data base that semantic model used to connect. layers: Workspace--> report--> semantic model--> data source(eg: AAS model). I can get list till 3 layers but unable to get details of 4th layer(data source) using admin monitoring folder data. I have fabric and platform admin access. could you please help me to know if there is a way to extract complete details in power bi service. Unfortunatly, I don't have access to run API's directly, your support helps to get the detailed list of all the workspaces in power BI service.94Views1like4CommentsPowerBI dataflow not visible in PowerBI desktop
On platform workspace I have multiple dataflows, all showing up and working when I try to connect to them from desktop app. yesterday I created new dataflow, same workspace, updated data, when I click on it in the platform it shows my data. But when I try to connect to it from the app it does not show up. all other dataflows are visible. gen1. what can be the problem?Solved91Views0likes4Comments