Scheduled refresh and Manual refresh Error
Hi i hope someone can help me to fix the issue regarding the refresh on my report, This report has been running for a year now suddenly 3 days ago errors keeps popping out. Thanks Data source error: DataSource.Error: OData: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.. Microsoft.Data.Mashup.ValueError.DataSourceKind = Dynamics365BusinessCentral. DataSourcePath = Dynamics365BusinessCentral. . The exception was raised by the IDataReader interface. Please review the error message and provider documentation for further information and corrective action. Cluster URI: Activity ID: Request ID: Time: 2026-10-09 16:30:35Z Details # Type Start End Duration Status Details 1 Data 10/10/2026, 12:15:21 AM 10/10/2026, 12:17:18 AM 1m 57s Failed (Show) 2 Data 10/10/2026, 12:18:18 AM 10/10/2026, 12:20:31 AM 2m 12s Failed (Show) 3 Data 10/10/2026, 12:22:31 AM 10/10/2026, 12:24:03 AM 1m 31s Failed (Show) 4 Data 10/10/2026, 12:29:03 AM 10/10/2026, 12:30:35 AM 1m 32s Failed (Show)
Moving finance Excel reporting to Power BI: how would you handle versioning and RLS?
Hi all, I’d like to ask how you’d handle a finance reporting setup where the source is still Excel, because I’m not sure we landed on the best design. The sources are multiple Excel files with budget, cost and revenue data, maintained separately for different regions. The problems were the usual ones: combining the files led to duplication and inconsistencies, several versions of the same file were circulating, and reconciling everything took a lot of manual effort. Budget vs. actuals and profitability trends were hard to track, and sensitive financial data was being shared without any role-based access. For the fix, we set up predefined formats, validation checks and version control on the Excel inputs, then used Power Query for the ETL, with dataflows handling integration and versions. The financial datasets are stored in SharePoint/Cloud behind role-based access, with the Data Gateway for connectivity. In Power BI we used hybrid data modeling with automated refreshes, row-level security, deployment pipelines and audit logs for governance. The reports cover variance analysis, KPIs and trend lines across regions. So here’s what I keep wondering. When the inputs are still Excel files owned by finance teams, how strictly do you enforce a template at the source? Do you reject files that fail validation, or clean them up downstream in Power Query? For version control, are you relying on SharePoint versioning, dataflow versions or something else? And how do you stop someone from still emailing around a “final_v3” file next to the governed report? On the model side, we went with hybrid modeling. How do you decide what to import and what to leave in DirectQuery or a composite model, especially for budget vs. actuals? And with RLS across regions, how do you manage the roles as the number of regions and stakeholders grows? Do you map roles through security groups, or maintain a mapping table in the model? Last one: deployment pipelines work fine for the reports, but has anyone found a clean way to handle dataflow and parameter changes between stages without breaking the gateway connections? Would love to hear how you’d approach it.Right-sizing Fabric capacity with 11 models and 47 dataflows: where do you stop?
Hi all, I'd like to ask how you'd approach a Power BI and Fabric environment that has grown past what the capacity can comfortably handle, because I'm not sure we found the best way through it. The environment has been growing since 2021 and now has 11 complex semantic models (sales, production, job costing and finance) fed by 47 dataflows. Sources are an Odoo ERP hosted on AWS, Google Sheets and Google Drive, with ingestion through Power Automate. Over time the symptoms piled up: slow refreshes, heavy load on the server, and Fabric consumption that was higher than it should have been. On the governance side there were shared logins, weak role-based controls and unstructured workspaces. We started with an audit of every model, dataflow, governance setting and the Fabric usage. Then we consolidated fragmented models to cut redundancy, redesigned the dataflows, and moved to incremental refresh with query folding where possible. For governance we added RBAC, a Dev/Test/Prod workspace structure, sensitivity labels and lineage tracking, plus a certified semantic model and quarterly audits. After the optimization, we were able to bring capacity down from 64 CU to 16 CU, and in some cases 8 CU. What I'm curious about is how you decide where to stop. How do you know how far you can go on right-sizing before refresh windows or interactive queries start to suffer? Do you look at the Capacity Metrics app over a few weeks, or do you test by dropping the SKU and watching? And how much do you rely on smoothing and bursting? With 47 dataflows, would you keep them as dataflows, or move some of the heavier logic into a Lakehouse or Warehouse? And with sources like Google Sheets and an ERP, how do you confirm query folding is actually happening and not silently falling back? On model consolidation, did you go for fewer, bigger models or a central certified model with thin reports? What did that do to refresh times and to your DAX? Last one: for the Dev/Test/Prod setup, are you using deployment pipelines, or something else? Has anyone found a way to keep dataset and dataflow parameters from breaking between stages? Would love to hear how you've handled this.
F64 Import Model Refresh Fails Due to Memory: Can Data Duplication Be Avoided
Hello, We are experiencing refresh failures for a large Power BI Import semantic model on an F64 Microsoft Fabric capacity. The refresh fails because the operation exceeds the available memory. Our main objective is to resolve the refresh failure. We are also trying to understand how the model is compressed and which objects consume the most storage, because reducing the model size may help reduce the refresh-time memory requirement. Current Environment The semantic model contains three fact tables and multiple dimension tables. The data source is an Azure Synapse Analytics dedicated SQL pool. The semantic model and reports run on an F64 Microsoft Fabric capacity. The semantic model uses Import storage mode. The semantic model size is approximately 14.8 GB. We import only the columns required for reporting, relationships, calculations, and security. One fact table contains approximately 3.8 billion rows. This large fact table is divided into partitions of approximately 100 million rows using XMLA. We refresh only the partitions that contain changed data. The other two fact tables have been denormalized by adding only the required descriptive attributes that were previously obtained from dimension tables. We denormalized those two fact tables because some table visuals were failing with the following error when using the normalized design: Query exceeded available resources. The denormalized design improved the table visual behaviour, but it increased the semantic model size. Main issue: Refresh failure due to memory usage Even though we refresh only the affected partitions of the 3.8-billion-row fact table, the semantic model refresh is failing because of the memory constraint on the F64 capacity. We would like to understand how memory is used during an Import semantic model refresh. Our understanding is that Power BI needs to preserve the currently queryable version of the model or partition while it processes a new version. This could temporarily require memory for both existing and newly processed data, along with additional memory for compression, dictionaries, relationships, indexes, and transaction commit operations. Could you please clarify the exact refresh behaviour? Specifically: Does Power BI temporarily maintain two copies of the complete semantic model, or only two versions of the partition being refreshed? Which model structures may be duplicated or rebuilt during a partition refresh? How much additional working memory is normally required beyond the stored semantic model size? Could a 14.8 GB semantic model exceed the F64 memory limit during refresh because the existing model, refreshed partition, dictionaries, relationship indexes, and processing workspace are held in memory at the same time? Does refreshing one partition at a time materially reduce peak memory, or can model-level structures still cause high memory consumption? Can refresh-time model duplication be disabled? Is there any supported way to prevent or reduce the apparent duplication of data during refresh? For example, can an Import refresh directly modify or update the existing data in a partition instead of building a separate replacement version and switching to it after processing completes? We would like to know whether any of the following is possible: Update existing compressed data in place. Append new rows directly to an existing processed partition. Delete or modify individual rows without recreating the affected partition. Disable the refresh transaction or shadow-copy behaviour. Disable the retention of the old partition version during processing. Commit the refresh in smaller stages to reduce peak memory. Release the old partition from memory before loading the replacement. Process only model metadata, relationships, or calculations without creating another data copy. Configure a lower-memory refresh mode for large Import semantic models. Control the number of rows or segments processed in each refresh transaction. If in-place updates or disabling refresh-time duplication are not supported, what is the recommended approach for refreshing a model of this size on F64? We understand that transactional refresh behaviour may be required to keep the existing semantic model available to report users and to allow rollback if processing fails. However, we would like confirmation of whether this behaviour can be changed or optimized. VertiPaq dictionary and compression question We are also investigating which columns contribute most to the semantic model size. Consider the following simplified example: LargeFact[EntityIdentifier] SecondFact[EntityIdentifier] EntityDimension[EntityIdentifier] All three columns use the same data type and contain many of the same identifier values. Does VertiPaq create: one shared dictionary for the identifier across the complete semantic model, one dictionary per table, one independent dictionary per physical column, or separate dictionaries or encoding structures at the partition or segment level? Does the relationship between the fact and dimension tables allow the key dictionaries to be reused, or does every physical column maintain its own dictionary and encoded value storage? For example, suppose the fact table containing 3.8 billion rows has a foreign-key column referencing another table. If that foreign-key column contains substantially fewer distinct values than the total row count, is its storage broadly composed of: a dictionary containing the distinct values, encoded references for the 3.8 billion rows, column segments, relationship indexes, hierarchy structures, and other internal storage objects? We would like to understand whether the storage is primarily caused by: the dictionary, the encoded values across 3.8 billion rows, internal relationship structures, partition-related structures, or a combination of these. Storage query used I used the following DAX query to identify the columns and internal storage objects consuming the most space: // Check largest storage entity EVALUATE VAR SegmentSizes = GROUPBY ( INFO.STORAGETABLECOLUMNSEGMENTS(), [TABLE_ID], [COLUMN_ID], "Records", SUMX ( CURRENTGROUP(), [RECORDS_COUNT] ), "SegmentUsedBytes", SUMX ( CURRENTGROUP(), [USED_SIZE] ), "SegmentAllocatedBytes", SUMX ( CURRENTGROUP(), [ALLOCATED_SIZE] ) ) VAR ColumnDetails = SELECTCOLUMNS ( INFO.STORAGETABLECOLUMNS(), "TABLE_ID", [TABLE_ID], "COLUMN_ID", [COLUMN_ID], "Table", [DIMENSION_NAME], "Column", [ATTRIBUTE_NAME], "ColumnType", [COLUMN_TYPE], "DataType", [DATATYPE], "Encoding", [COLUMN_ENCODING], "DictionaryBytes", [DICTIONARY_SIZE] ) VAR Combined = NATURALLEFTOUTERJOIN ( SegmentSizes, ColumnDetails ) RETURN SELECTCOLUMNS ( Combined, "Table", [Table], "Column or object", [Column], "Object type", [ColumnType], "Data type", [DataType], "Encoding", [Encoding], "Records", [Records], "Segment MB", DIVIDE ( [SegmentUsedBytes], 1024 * 1024 ), "Dictionary MB", DIVIDE ( [DictionaryBytes], 1024 * 1024 ), "Approximate total MB", DIVIDE ( [SegmentUsedBytes] + COALESCE ( [DictionaryBytes], 0 ), 1024 * 1024 ), "Allocated MB", DIVIDE ( [SegmentAllocatedBytes], 1024 * 1024 ) ) ORDER BY [Approximate total MB] DESC The query showed identifier columns from multiple tables among the largest storage objects. The most significant result was a foreign-key column in the 3.8-billion-row fact table. The query reported that this column, or its related internal storage object, was consuming up to approximately 13 GB. Could you please confirm whether this query correctly estimates storage usage by column or internal storage object? Main questions Our primary questions are: How can we resolve the refresh failure caused by the F64 memory constraint? Does Import refresh temporarily duplicate the complete model, or only the affected partition and related structures? Is there any supported way to update an existing Import partition in place? Can refresh-time duplication or transactional processing be disabled? Is there a supported configuration that reduces peak refresh memory? Is the storage query above correct, particularly the approximately 13 GB reported for a foreign-key object? Are VertiPaq dictionaries maintained per column, per table, per partition, or per model? What architecture would Microsoft recommend for an Import semantic model containing a 3.8-billion-row fact table on F64? Our immediate objective is to complete the refresh successfully. Model-size optimization is important mainly because it may reduce the peak memory required during processing. At the same time, we need to avoid reintroducing the 'Query exceeded available resources.' error in the report table visuals. Thank you.
Fabric Data Agent Access issue
I have created a fabric data agent using semantic model of a published report, It is working fine but I'm not able to share it with others I published the agent in M365 copilot and share with my teams of 100 people only 4 or 5 people can go to M365 copilot and search the agent and added the agent to their agent tab others are not even seeing the agent but if I share the agent link from M365 copilot then they see the agent but agent is not answering the questions but it's perfectly answering for me and other 4 to 5 people who see the agent when I surfed it said it said there are 2 level of access to get answer from the agent 1. agent level access 2. underlying data level access I have workspace contributor access for workspace, read, write, reshare permission for the agent. most of them share the same access but there are not able to see the only difference I found is I'm having premium per user license and most of other have pro license so I doubted it but internet says license is not an issue can anyone pls help me with this issue
One of my Dataflow Gen 1 stopped working
Hello everyone, I have a few DataFlow Gen1, all of them are working except one. "TauxHorairePanierMoyen" worked perfectly for months, and this morning I can't update the file. It shows this kind of error : Error: Request ID: 82fd0734-bf12-4ff2-b85d-79e2981cb408 Activity ID: 0b7f17c0-b8c4-4f96-beef-896adb689b04 When I go to Power Query it loads perfectly, but when I update, it doesn't work. --> It's a combined Excel Files. Requested on Nom du flux de données État d'actualisation du flux de données Nom de la table Nom de partition Actualiser l’état Heure de début Heure de fin Durée Lignes traitées Octets traités (Ko) Validation maximale (Ko) Temps processeur Temps d’attente Moteur de calcul Erreur 30/09/2026 11:29 TauxHorairePanierMoyen Échec TauxHorairePanierMoyen Non disponible Échec 30/09/2026 11:29 30/09/2026 11:35 00:06:00.7300 Non disponible Non disponible Non disponible Non disponible Non disponible Non disponible Error: Request ID: 82fd0734-bf12-4ff2-b85d-79e2981cb408 Activity ID: 0b7f17c0-b8c4-4f96-beef-896adb689b04 My Power Query code : Source = SharePoint.Files("XXX", [ApiVersion = 15]), #"Lignes filtrées" = Table.SelectRows(Source, each Text.Contains([Name], "TauxHorairePanierMoyen")), #"Fichiers masqués filtrés" = Table.SelectRows(#"Lignes filtrées", each [Attributes]?[Hidden]? <> true), #"Appeler une fonction personnalisée" = Table.AddColumn(#"Fichiers masqués filtrés", "Transformer le fichier", each #"Transformer le fichier"([Content])), #"Colonnes renommées" = Table.RenameColumns(#"Appeler une fonction personnalisée", {{"Name", "Source.Name"}}), #"Autres colonnes supprimées" = Table.SelectColumns(#"Colonnes renommées", {"Source.Name", "Transformer le fichier"}), #"Colonne de table développée" = Table.ExpandTableColumn(#"Autres colonnes supprimées", "Transformer le fichier", Table.ColumnNames(#"Transformer le fichier"(#"Exemple de fichier"))), #"Type de colonne changé" = Table.TransformColumnTypes(#"Colonne de table développée", {{"Nom Affaire", type text}, {"ORV", type text}, {"Nature ORV", type text}, {"Code équipe", type text}, {"Libellé équipe", type text}, {"N° client", type text}, {"Nom client", type text}, {"Nature (PGC)", type text}, {"Type ligne (PR/MO/DIV)", type text}, {"Date facture", type date}, {"No facture", Int64.Type}, {"Libellé réceptionnaire", type text}, {"Référence", type text}, {"Désignation", type text}, {"Quantité servie", type number}, {"Montant net", type number}, {"Montant brut", type number}, {"Montant remise", type number}, {"Montant remise CBQ", Int64.Type}, {"Remise client (Code)", type text}, {"Libellé remise CBQ", type text}, {"Famille comptable Reconstitué (Code)", type text}}) in #"Type de colonne changé" File Exemple : let Source = SharePoint.Files("XXX", [ApiVersion = 15]), #"Lignes filtrées" = Table.SelectRows(Source, each Text.Contains([Name], "TauxHorairePanierMoyen")), #"Fichiers masqués filtrés" = Table.SelectRows(#"Lignes filtrées", each [Attributes]?[Hidden]? <> true), Navigation = #"Fichiers masqués filtrés"{0}[Content] in Navigation Parameter let Source = Excel.Workbook(Paramètre, null, true), Navigation = Source{[Item = "TauxHoraire&PanierMoyen", Kind = "Sheet"]}[Data], #"En-têtes promus" = Table.PromoteHeaders(Navigation, [PromoteAllScalars = true]) in #"En-têtes promus" Transform File Exemple : let Paramètre = #"Exemple de fichier" meta [IsParameterQuery = true, IsParameterQueryRequired = false, Type = type binary, BinaryIdentifier = #"Exemple de fichier"] in Paramètre Thanks for your help. Kinds regards, Aude-Marie
