cicd
456 TopicsIssue during the deployment of a WH table from dev to test workspace
Hi, in a my Fabric solution, I've developed a data warehouse in a wh, existing also in a test workspace. I need to change the structure of some tables, by altering or adding columns. When I try to deploy the changed wh tables from the dev to test workspace using a deployment pipeline I've an error about an alter column issue. Now, how could I execute the changed tables deployment rightly? Is there a Fabric bug? I need to accomplish the deployment actions without any problems. Thanks38Views0likes3CommentsUnable to Configure Deployment Rules for a Direct Lake Semantic Model in Fabric
Hi Fabric Community, I’m facing an issue with Fabric Deployment Pipelines and would appreciate some guidance. I have a Power BI semantic model using Direct Lake mode, and I’m deploying it between two Microsoft Fabric workspaces. I am an Admin in both the source and target workspaces. However, when I open the Semantic model deployment rules, both Data source rules and Parameter rules are disabled/grayed out, as shown in the screenshots. I also confirmed that the tables in the semantic model are using Direct Lake storage mode using the TABLETRaits() DAX query. My questions are: Why are Data source rules and Parameter rules disabled for this semantic model? Is there any additional configuration required to enable these deployment rules? If deployment rules are not supported for this scenario, what is the recommended approach for changing the data source between environments? Any advice or explanation would be greatly appreciated. Thanks in advance!75Views0likes6CommentsFabric Warehouse Git commit fails with “Maximum commit size exceeded” potentially due to XMLA.json
Based on the Fabric Warehouse Git integration documentation, the XMLA.json file should be excluded from Git integration workflows. Fabric should exclude this file from commits and updates to prevent the default semantic model metadata from being unintentionally stored in Git. [learn.microsoft.com] However, it appears that XMLA.json, or the associated semantic model metadata, may still be getting processed on the Fabric side while calculating or preparing the commit. This potentially causes the commit to exceed the maximum allowed size. Any One Face Same Issue, how to avoid above issue & prevent XMLA.json file from getting processes while git commit ?29Views0likes4CommentsHow to implement CI/CD for Lakehouse Delta table schemas and SQL Analytics Endpoint
Hi Microsoft Fabric Community, We have set up an automated CI/CD pipeline using Azure DevOps Repos, Git integration, and Microsoft's fabric-cicd Python library. Our current workflow is: 1. Feature workspaces are connected to Git feature branches. 2. A Pull Request merges changes into the dev branch. 3. Our Azure DevOps pipeline deploys Fabric items across target workspaces (dev → test → prod). The current behavior is: Control-plane items such as Notebooks, Data Pipelines, and Copy Jobs deploy and parameterize successfully. The Lakehouse item itself and its OneLake shortcut definitions also deploy as expected. However, Delta table schemas under /Tables and SQL Analytics Endpoint objects such as Views, Stored Procedures, and TVFs are not captured or deployed. Git integration appears to track the Lakehouse metadata but not the underlying Delta table structures or SQL Analytics Endpoint DDL. We have the following questions for the product team and community: Question 1 - Lakehouse Delta Table Schemas How can we implement automated CI/CD for Delta table creation and schema evolution across environments without requiring manual execution? Question 2 - SQL Analytics Endpoint Objects Since DACPAC / SqlPackage does not support the read-only SQL Analytics Endpoint, what is the recommended approach for implementing CI/CD for Views, Stored Procedures, and TVFs using Azure DevOps? Question 3 - Fabric Roadmap Are there plans on the Microsoft Fabric roadmap for native Git integration to track and synchronize SQL Analytics Endpoint objects directly alongside other workspace items? Any reference architectures, repository templates, recommended tools, or best practices for implementing this type of CI/CD workflow would be greatly appreciated. Thank you!87Views0likes6CommentsGit exception should be more specific
I was unable to pull updates from git, with the below screenshot: But it's hard to verify which code paragraph is having issues, which made me hard to identify the issue. would that be possible to improve the error message to something like: Thanks!47Views1like6CommentsGit Commit Issue
Hello Fabric Community, I have a workspace that is connected to my GitHub repo. I used a personal access token to integrate GitHub with Fabric. Initially, I created the token for only one day and committed my items to GitHub. When I connected my GitHub repo again with a new token, I was unable to commit the changes that had been made so far because I could not find any items pending to commit in the source control pane "Changes" tab. PFA. Please suggest a solution or let me know the proper way to integrate my changes into GitHub again. Thanks in advance for any insights or guidance!53Views0likes4CommentsDP-600 Voucher
a { text-decoration: none; color: #464feb; } tr th, tr td { border: 1px solid #e6e6e6; } tr th { background-color: #f5f5f5; } Hi everyone, I recently passed the DP-700 Microsoft Fabric Data Engineer Associate certification and am now planning to take DP-600 (Microsoft Certified: Fabric Analytics Engineer Associate) while the Fabric concepts are still fresh in my mind. Unfortunately, I wasn't able to use a previous DP-600 voucher when I first received it as I wasn't ready for the exam at the time. I was wondering if anyone is aware of any current voucher opportunities for DP-600, or has an unused voucher that they won't be using and is transferable (if permitted by the voucher terms). Any guidance would be greatly appreciated. Thanks in advance! Regards, Amanul65Views1like2CommentsMicrosoft Fabric Materialized Lake Views (MLVs) – Looking for Confirmation on a Few Points
Microsoft Fabric Materialized Lake Views (MLVs) – Looking for Confirmation on a Few Points Hi everyone, I'm currently working with Materialized Lake Views (MLVs) in Microsoft Fabric and trying to establish the right approach for implementing them in a proper Dev → UAT → Production setup. I have gone through the available documentation, but I would appreciate confirmation from people who are already using MLVs in real projects. MLV Refresh Failure and Monitoring My current understanding is that if an MLV fails during its initial execution or during a scheduled refresh: The MLV run will be marked as Failed. The failure can be investigated through the MLV run history/monitoring experience. The error code, error message and execution details can be used for troubleshooting. If an upstream MLV fails, dependent MLVs may be skipped. Questions: Is there a recommended/native way to send an email or Teams notification whenever an MLV refresh fails? What is the recommended production monitoring approach for MLV refresh failures? Is anything else required in addition to configuring the MLV schedule? ON MISMATCH FAIL vs ON MISMATCH DROP My current understanding is that ON MISMATCH is related to handling schema/structure mismatches encountered during MLV processing, rather than being a general data-quality validation mechanism. For example: ON MISMATCH FAIL would cause the MLV operation to fail when the applicable mismatch occurs, whereas: ON MISMATCH DROP would allow processing to continue while dropping the affected/incompatible data. Questions: Is this understanding correct? What exactly constitutes a "mismatch" for an MLV? What are the recommended scenarios for using FAIL vs DROP? Does ON MISMATCH FAIL affect the entire MLV refresh or only the affected data/operation? Dev → UAT → Production Deployment This is probably my biggest area of uncertainty. Suppose I create and test the following in DEV: DEV Workspace Lakehouse Source tables MLV 1 MLV 2 My expectation is that I should not manually recreate the MLVs in UAT and Production. Instead, I would use Fabric lifecycle/deployment capabilities to promote the artifacts: DEV → UAT → PROD Questions: Are MLV definitions currently supported for this type of Dev → UAT → Prod deployment? Does Deployment Pipeline automatically handle the corresponding Lakehouse/item dependencies? If the workspace names, Lakehouse names and item IDs are different between environments, how are those references handled? Does Fabric automatically rebind supported dependencies to the corresponding target-environment resources? Which MLV-related properties/configurations need to be manually configured or validated after deployment? For example: DEV Workspace: BI_DEV Lakehouse: Sales_LH_DEV ↓ UAT Workspace: BI_UAT Lakehouse: Sales_LH_UAT ↓ PROD Workspace: BI_PROD Lakehouse: Sales_LH_PROD How much of this environment-specific mapping is automatically handled by Fabric? MLV Refresh Schedules Across Environments Suppose my DEV MLV has a refresh schedule of every 1 hour. When I deploy it to UAT and Production: Is the schedule also deployed? If it is, can or should it be different between environments? What is the recommended approach for managing MLV schedules separately in DEV, UAT and PROD? Is schedule configuration considered part of the deployable MLV artifact or environment-specific configuration? MLV Dependencies / MLV on top of MLV Can we create an MLV that references another MLV? For example: Source Tables → MLV 1 – Silver → MLV 2 – Gold → MLV 3 – Aggregation Is this a supported and recommended architecture? If yes: Are there any limitations on the number or depth of MLV dependencies? How does refresh orchestration work? If MLV 1 fails, are MLV 2 and MLV 3 automatically skipped? How is the dependency and lineage handled during deployment from DEV → UAT → PROD? SQL View on top of MLV Can we create a regular SQL View on top of an MLV? For example: Source Tables → Materialized Lake View → SQL View → Semantic Model / Power BI If this is supported: Are there any performance implications? Does the SQL View remain virtual while the MLV remains materialized? Is this a recommended pattern for exposing MLV data to downstream consumers? View → MLV Conversely, can an MLV be created on top of a regular SQL View? For example: Source Tables → SQL View → MLV If supported, are there any restrictions around the types of views or queries that can be used as an MLV source? Recommended Production Architecture For a production implementation, would the following architecture be considered a reasonable pattern? Source Tables → MLV Layer → Silver MLVs → Gold MLVs → SQL Views (if required) → Semantic Model → Power BI Or is it generally better to consume MLVs directly from the semantic model where possible? General Best Practices For anyone using MLVs in production: What are the main limitations you have encountered? What should be considered when designing MLV dependencies? How do you handle monitoring and alerting? How do you manage Dev/UAT/Prod deployment? Are there any MLV-specific CI/CD considerations that are easy to miss? What would you recommend avoiding when designing an MLV architecture? I'd particularly appreciate confirmation from anyone who has implemented MLVs across multiple Fabric environments such as DEV, UAT and PROD. Thanks in advance!Solved104Views2likes5CommentsFabric Data Pipeline – Does Office 365 Outlook Activity Support Service Principal Authentication?
Hi everyone, I’m looking for confirmation from anyone who has successfully implemented **Service Principal authentication with the Office 365 Outlook activity in a Microsoft Fabric Data Pipeline**. We currently use the **Office 365 Outlook activity**, authenticated with a personal user's OAuth login. This causes problems with CI/CD because the connection does not reliably carry over between workspaces/environments. We’re therefore looking at moving to **Service Principal authentication**, which now appears as an option in the connector. We have created an App Registration with: * Microsoft Graph * `Mail.Send` – Application permission Before granting admin consent, I’d like to clarify a few things. **1. Does Service Principal authentication actually work end-to-end with the Office 365 Outlook activity?** Has anyone successfully configured this in Fabric and used it across multiple workspaces/environments through CI/CD? There are previous discussions suggesting that although Service Principal authentication appears in the UI, it may not actually work with the Outlook activity. https://community.fabric.microsoft.com/discussions/ac_dataengineering/how-to-setup-service-principal-for-microsoft-outlook-365-connector/5189483 If you have this working, it would be great to know: * What configuration/permissions you used * Whether admin consent was required * Whether the connection survives deployment between workspaces * Whether the Outlook activity successfully sends emails using the Service Principal **2. Can the `Mail.Send` permission be restricted?** `Mail.Send` as an Application permission is quite broad. Is there a supported way to restrict the Service Principal to specific mailbox(es), for example using **Exchange Online RBAC for Applications**? **3. Is Microsoft Graph via an HTTP/Web activity a better option?** If Service Principal authentication with the built-in Outlook activity is unreliable, would calling Microsoft Graph `sendMail` through an HTTP/Web activity be a more reliable and least-privilege approach for CI/CD? If anyone has this working in **production or across multiple Fabric environments**, I’d really appreciate details of the configuration and any limitations you encountered.Thanks in advance.Solved95Views2likes6Comments