If you are working with large fact tables in Fabric, loading everything on every run quickly becomes slow and costly. Incremental loading fixes that by moving only the new records since the last pipeline run, and the watermark pattern makes this easy to manage. In this walkthrough, you will see how to set up a simple watermark table, build a stored procedure, and wire up a Fabric pipeline that reads the last load timestamp, fetches the latest data from FactImport, copies only the delta, and updates the watermark for the next run. It is a straightforward pattern that keeps your Warehouse and Lakehouse in sync without wasting compute or time.

Power Query’s Table.TransformColumnTypes has long supported a Culture tag as an optional third parameter. As of May 2025, it now goes further: you can pass an Options record that includes both Culture and MissingField. This makes it the only M function where MissingField lives inside the Options record, bringing new flexibility and fewer errors.

DISCLAIMER: This method uses the internal takeout-pa.clients6.google.com API and is thus UNSUPPORTED and BRITTLE.

This is a custom function that you can paste into advanced editor and then use to get Public Google Drive contents with just the Drive ID and without needing a) any authentication or b) a Google Cloud Project key.

tl;dr List.Contains is slow in Power Query because for each iteration, for example each row in your table you check, it scans the list from the start until it finds a match. Buffering helps so it doesn’t re-evaluate on every iteration, but it still scans each item in the list. A better approach is merging via Table.Join, or best of all (in the right scenario), turn your list into a single record to create a hash map and do near constant-time lookups with Record.Fi eldOrDefault.

How to Build and Orchestrate a Scalable Data Pipeline in Microsoft Fabric? A Step-by-Step Guide

In today’s data-driven world, organizations handle massive volumes of information from multiple sources. The challenge is not just storing this data but ensuring it is organized, transformed, and analytics-ready—without manual intervention.

Auto-Magic Data Ingestion in Microsoft Fabric Lakehouse using Data Activator & Pipelines

Imagine this you drop a .csv file into your Lakehouse, and within moments - without any manual intervention - it appears as a fully formed Delta Table in your workspace. No scripts, no button clicks. Just pure automation.

Learn a handy Power BI trick to set column data types directly in the Power Query M formula. This approach avoids adding extra “Changed Type” steps and keeps your queries clean.

Ever wondered if you can send personalized emails directly from Microsoft Fabric Data pipelines?
In my latest blog, I show how I built a simple automation that notifies employees when their weekly working hours fall below a threshold — no Power Automate, no external tools.
All done within the Fabric ecosystem

Public parameters are two small words that substantially boost the versatility and usability of Dataflow Gen2s in Microsoft Fabric for your data orchestrations.

In this blog post, we'll explore how you can start using Fabric Data Pipelines as a Power BI user that wants to take full advantage of Microsoft Fabric. And if you have never used Power BI before but still want to start with pipelines, you are in the right place!

In this blog we discuss how to use the expression language to handle referencing a field that may or may not exist at runtime, a non-existent property. 

This short blog details a common scenario we saw in Azure Data Factory where we wanted to ignore zero-byte (empty) files landing in out storage accounts.  In this blog we show you how to achieve this functionality with data pipeline storage event triggers (preview) and provide references to the properties and schemas of the event grid topics which will empower you to specify the filters you need to be successful. 

For the last few days, I have been working on the Contoso Sales data to create a Power BI report as part of the learning. Currently, I am using the default ready to go Power BI data model provided by Microsoft which can be found here. As Microsoft Fabric is the new tech buzz, so I thought why don’t I get this data somehow in the Fabric environment.

In this post I want to cover my initial tests of the Data Factory Testing Framework. Which is a unit testing framework you can use to test Microsoft Fabric Data Pipelines.

I wanted to cover this framework since I mentioned it in a previous post about unit tests on Microsoft Fabric items.