From Chaos to Clarity: How the Manufacturing Dashboard Helps Everyone Understand the Business
Let's walk through what each page actually shows, and why it matters. Page 1: Customer Insights — "Who are we selling to, and are they happy?" Every business lives or dies by its customers, but it's surprisingly hard to keep track of hundreds of relationships in your head. This page acts like a customer relationship "scoreboard." It shows things like: * How many active customers the business currently has * Which customers are considered healthy (buying regularly, paying on time) versus at risk (going quiet, showing warning signs) * How revenue is spread across different customers and regions — are we relying too heavily on just a few big accounts? Think of it like a doctor's checkup, but for relationships instead of health. If ten customers suddenly go from "healthy" to "at risk," that's a signal to act before they walk away for good — not after the revenue has already dropped and everyone's asking why. Page 2: Production & Supply — "Are we making enough, and can we deliver it?" This is the page for anyone who cares about what's actually happening on the factory floor and getting product out the door. It's less about money and more about operations. Key questions it answers: * How much did we produce this month, and is that on target? * Are shipments going out on time, or are we falling behind on delivery promises? * Is one particular plant underperforming compared to the others? If you're a plant manager, this is likely the first thing you check every morning — before coffee, even. It tells you at a glance whether today is a "business as usual" day or a "we need to fix something now" day. Page 3: Overview — "How's the whole business doing, in 30 seconds?" This is the page built for someone with almost no time to spare — a CEO, an investor, or anyone who just needs the headline numbers without digging through details. At the top sit eight simple cards, each showing one important number and whether it went up or down compared to last month: * Total Revenue — how much money came in * Total Production — how much product was made * Capacity Utilization — how much of the factories' potential is actually being used * Gross Profit — how much money is left after production costs * Supply Fulfillment — how reliably orders are being delivered * Inventory Value — how much stock is sitting in the warehouse * Active Customers — how many customers are currently buying * Overall Operational Efficiency — a single score summarizing how smoothly everything is running Below that, charts break revenue and production down by month, by plant, and by region — so if a number drops, you don't just see that it dropped, you can immediately see where. Was it one plant having a bad month, or a slowdown across an entire region? There's also a simple "Alerts & Insights" section that puts the numbers into plain words — things like "Supply on track: fulfillment is 94% and improving" — so nobody has to guess what a chart is trying to tell them. Page 4: Inventory & Working Capital — "Do we have too much stock, or too little?" This page tackles a balancing act every manufacturing business faces. Keep too much inventory sitting around, and you're tying up cash and warehouse space that could be used elsewhere. Keep too little, and you risk running out of product right when a customer needs it — losing sales and trust. This page shows: * Total Inventory Value — how much money is currently tied up in stock * Inventory Turnover — how quickly that stock is being sold and replaced (a higher number generally means things are moving efficiently) * Days Inventory Outstanding — roughly how many days' worth of stock is sitting around unused * Raw Material vs. Finished Goods Stock — how much is still waiting to be turned into product versus ready to ship * Slow-Moving Inventory — stock that isn't selling and may need attention * Stockout Risk — a warning flag for items at risk of running out * Inventory Accuracy — how well the recorded stock counts match what's physically in the warehouse There's even a detailed table listing specific materials by name, plant, and status (like "Slow-Moving" or "Healthy"), so instead of a vague warning, someone gets a precise to-do list of exactly what needs a closer look. Why This Approach Works for Everyone The real magic of this dashboard isn't any single chart — it's the consistency. Every page follows the same basic structure: 1. Filters at the top (date range, region, plant, product category) so anyone can narrow the view to exactly what matters to them 2. A handful of key numbers, shown as simple cards with an up or down arrow — no complicated formulas to interpret 3. A plain-language "Alerts & Insights" section that explains, in a sentence or two, what changed and why it matters That means a machine operator, a plant manager, a customer success rep, and a CEO can all open the same dashboard and immediately find what's relevant to them — no translation needed, no waiting for someone else to "run the numbers." In a business with as many moving parts as manufacturing, that kind of shared clarity isn't a luxury. It's what keeps everyone — from the shop floor to the boardroom — pointed in the same direction.Is Your Semantic Model Ready for Copilot? A Fabric IQ Readiness Guide
For years, the Power BI semantic model had one main job: feed reports and dashboards. That job has changed. With Fabric IQ, your semantic model is becoming the place Microsoft 365 Copilot and your AI agents get their business answers from. If the model is messy, inconsistent or badly documented, Copilot's answers will be too. This guide explains what Fabric IQ is, why it matters, what it costs, and how to get your models ready in about 30 days. What is Fabric IQ? Fabric IQ is the semantic foundation over your structured business data. It is one of the four pillars of Microsoft IQ, was announced at Build 2026, and is now generally available. It brings three things together: your data in OneLake, the business meaning in your Power BI semantic models, and operational intelligence expressed as ontologies. Microsoft IQ is the umbrella. At Build, Microsoft presented it as the context layer that grounds AI agents in both general world knowledge and the organization's own knowledge. Work IQ covers people, emails, meetings and documents. Fabric IQ covers the numbers: revenue, margin, churn, inventory, and every KPI your business runs on. Medium Why this changes how people use data The biggest change is where people get their data answers. Instead of logging into Power BI to check a metric, business users can ask Copilot in Teams or Outlook a question in plain language and get an answer. Importantly, those answers come from your Power BI semantic models with existing row-level security and sensitivity labels applied. The reach keeps growing. Fabric IQ is integrated with Microsoft Agent 365 as a first-party MCP tool (in preview), and it is also extending into Microsoft 365 Copilot, including Cowork and Copilot Chat. Developers get it too: through Agent Skills for Fabric, Fabric IQ tools are available in GitHub Copilot CLI, so teams can query Power BI reports and semantic models from the command line. Microsoft Azure Put simply, one semantic model now answers questions in reports, in Teams, in Outlook, in custom agents and in the terminal. That is powerful when the model is good and risky when it isn't. This is Microsoft's stated direction, not a side feature. In a July 21, 2026 post on the Fabric Updates Blog about the future of conversational analytics, Microsoft described trusted business context in Fabric IQ that can be used consistently across Microsoft 365, Fabric, developer tools and future AI applications. The catch: reports have been hiding your model's gaps This is the point most teams miss. Fabric IQ skips the visualization layer and queries the model directly, so the model has to work on its own, outside the filter context of any specific report page. Think about how many of your reports quietly fix problems in the model: A page-level filter excludes test customers or cancelled orders A slicer defaults to "current fiscal year," so nobody notices the measure has no time logic A visual title explains what "Net Revenue" actually means Hidden columns and awkward field names only work because report authors know which ones to use When Copilot queries the model directly, none of those fixes apply. A user asks, "What was net revenue last quarter?" and gets a number that includes test data, the wrong calendar, or a measure nobody has used since 2023. Licensing: what you actually need You pay for this in two places: Microsoft 365 side: you need Microsoft 365 Copilot licenses to use Fabric IQ through Teams and Outlook. Fabric side: Fabric IQ grounding doesn't consume extra Fabric capacity from the Microsoft 365 side, but everything Fabric IQ grounds on still needs F-SKU capacity. For budgeting, this means heavier Copilot use doesn't directly raise your Fabric capacity bill for the grounding itself. You still need your models hosted on proper Fabric capacity, sized for the query load that conversational use brings. The 10-point readiness checklist Use this to review each semantic model before you expose it to Copilot. Clear business names. Rename tables, columns and measures in business language. fct_sls_amt_net should become Net Sales Amount. Copilot and your users both work better with plain names. Descriptions on everything that matters. Add descriptions to key tables, columns and measures. Say what each one means, what it includes and excludes, and the unit. This is the model's documentation for AI. One definition per metric. If you have Revenue, Revenue v2, Revenue (Finance) and Rev_Final, Copilot has to guess. Merge them into one certified measure and hide or remove the rest. Move report-level logic into the model. Any filter that must always apply, such as excluding test accounts or internal transactions, belongs in the model (in measures, RLS or the data itself), not on a report page. A proper date table. Mark it as a date table, include fiscal periods if your business uses them, and make sure time-intelligence measures work without a slicer. Hide the plumbing. Hide keys, technical columns, staging fields and helper measures. What stays visible is what Copilot considers fair game. Validate row-level security. Answers respect RLS, which is good. So test RLS properly with real user roles. A gap in RLS is now a gap in every Copilot conversation. Apply sensitivity labels. Label models that contain confidential or personal data so Microsoft's information-protection controls carry through to AI answers. Write AI instructions and version them. "Prep for AI" instructions are written on the semantic model, version-controlled with the model, and now shape Copilot's behaviour everywhere it grounds on that model. Use them to explain business rules, preferred measures and common terms (for example, "'customers' means active customers unless stated otherwise"). Test with real questions. Collect 20 to 30 questions your business users actually ask. Run them through Copilot and compare the answers with your certified reports. Fix any mismatch in the model, not in the report. A 30-day readiness plan Week 1: Inventory and prioritize. List your semantic models and choose two or three high-value, widely used ones, such as sales, finance or operations. Find each model's owner, check its current certification status, and identify which users will get Copilot access first. Week 2: Clean up the model. Work through checklist items 1 to 6: naming, descriptions, one version of each metric, moving report logic into the model, the date table, and hiding technical fields. This week delivers the biggest improvement in answer quality. Week 3: Security and AI instructions. Validate RLS against real roles, apply sensitivity labels, and write your first set of AI instructions. Put the model in source control (Fabric Git integration) if it isn't already, so instructions and changes are tracked. Week 4: Test, pilot and set up governance. Run your question set, fix the mismatches, then pilot with a small group of business users. Agree who owns each model, how changes get approved, and how users report a wrong answer. Governance: treat semantic models like products Once Copilot answers from your models, they are shared business infrastructure, no longer just a BI team's side project. A few habits help: Named owners for every model exposed to Copilot Certification so only reviewed models become the preferred sources Change control through Git and deployment pipelines, because a renamed measure can change hundreds of Copilot answers overnight Feedback loops so a wrong answer becomes a model fix within days, not months The bottom line Fabric IQ changes what a good semantic model is. It used to be a model that powered good reports. Now it has to answer business questions correctly on its own, in any tool, for anyone with permission to ask. Teams that clean up names, fix metric definitions, move logic into the model and write AI instructions now will have Copilot answers people trust. Teams that don't will spend 2027 explaining why Copilot and the dashboard disagree. Start with one model this month. The checklist above is enough to begin.Lakehouse vs. Warehouse in Microsoft Fabric: Which One Should You Actually Use?
If you're new to Microsoft Fabric, you've probably hit this wall already: you go to create an item, and Fabric hands you two very similar-sounding options - Lakehouse and Warehouse. Both store tabular data in Delta format in OneLake and provide SQL access, but they offer different development and transactional experiences. Both let you query with SQL.Fabric presents both as analytical data-store options, and their capabilities overlap enough that the choice is not always obvious. So which one do you pick? Short answer: it depends on who's writing the queries and what shape your data is in. Long answer: keep reading — and try the quick self-check below before you scroll to the recommendation. 60-Second Self-Check Answer these three questions honestly before you create your next Fabric item: 1. Who will write the transformation logic? - A) Python/Spark notebooks, data engineers comfortable with PySpark - B) SQL analysts and BI developers who live in T-SQL 2. What does your source data look like? - A) Mixed — JSON, Parquet, CSV, streaming events, semi-structured - B) Mostly clean, structured, relational-shaped data 3. What's the end consumption pattern? - A) A mix of ML, notebooks, ad-hoc exploration, and reporting - B) Primarily Power BI reports and governed semantic models Mostly A's → Lean Lakehouse. Mostly B's → Lean Warehouse. Mixed bag? → You're not alone — see the "Can I use both?" section below. Lakehouse: The Flexible One A Lakehouse stores data as files (Delta Parquet) in OneLake, and gives you two ways to work with it: - Notebooks (PySpark, Spark SQL) for engineers who want full control - A SQL analytics endpoint that auto-generates so SQL folks can still query the same tables Pick Lakehouse when: - Your data arrives messy, semi-structured, or in large volumes that benefit from Spark's distributed processing - Your team already thinks in notebooks and data science workflows - You want schema flexibility - Delta tables support schema enforcement and controlled schema evolution, offering flexibility while preserving data reliability. - You're building a medallion architecture (bronze → silver → gold) and need engineering muscle at the bronze/silver layers Watch out for: - The SQL endpoint is read-only - The SQL analytics endpoint is read-only for Lakehouse table data, so it does not support INSERT, UPDATE, or DELETE against those Delta tables. Modify or load the data through Spark or another supported ingestion and transformation experience. You can still create supported SQL objects such as views, functions, and stored procedures in the endpoint. - Fabric provides automatic Delta table optimizations, but advanced workloads may still benefit from deliberate file sizing, data layout, partitioning, or optimization strategies. (OPTIMIZE, V-Order, partitioning) python #Typical Lakehouse bronze-to-silver pattern in a notebook df = spark.read.format("json").load("Files/raw/events/") df_clean = df.dropDuplicates().withColumn("load_date", current_date()) df_clean.write.format("delta").mode("overwrite").saveAsTable("silver_events") Python from pyspark.sql.functions import current_date() df = spark.read.format("json").load("Files/raw/events/") df_clean = df.dropDuplicates().withColumn("load_date", current_date()) df_clean.write.format("delta").mode("overwrite") saveAsTable("silver_events") Warehouse: The Familiar One Fabric Warehouse provides a rich T-SQL-first data warehousing experience. — think of it as a cloud data warehouse that happens to store data in OneLake under the hood. Pick Warehouse when: - Your team's primary skill is T-SQL, not Spark/Python - You need full DML (`INSERT`, `UPDATE`, `DELETE`, `MERGE`) with transactional guarantees - You're modeling a governed, relational structure — star schemas, stored procedures, views - You want a more traditional data-warehouse development experience (cross-database queries, You want a familiar relational warehouse development experience with T-SQL, views, stored procedures, and support for compatible SQL tools.) Watch out for: - Less flexible with wildly semi-structured or streaming-first data — you'll typically land that in a Lakehouse first, then move it in Warehouse is designed primarily for T-SQL rather than Spark-native development. If Spark is central to your transformation logic, Lakehouse is usually the more natural starting point. sql -- Typical Warehouse transformation pattern MERGE INTO dbo.DimCustomer AS target USING staging.CustomerUpdates AS source ON target.CustomerID = source.CustomerID WHEN MATCHED THEN UPDATE SET target.Email = source.Email WHEN NOT MATCHED THEN INSERT (CustomerID, Email) VALUES (source.CustomerID, source.Email); Can I Use Both? Yes - and honestly, A supported and commonly discussed architecture is to use a Lakehouse for engineering-oriented layers and a Warehouse for a curated relational serving layer. do exactly this: - Lakehouse for bronze/silver ingestion and heavy transformation (Spark does the messy work) - Warehouse for the polished gold layer that analysts and Power BI consume with familiar T-SQL Both use OneLake, and Fabric supports patterns such as shortcuts and cross-database queries that can reduce or avoid unnecessary data duplication. If you physically load curated data into separate Warehouse tables, however, that creates another stored representation. Try It Yourself Before your next project kickoff, run this checklist with your team: [ ] Who owns the transformation code - engineers or SQL analysts? [ ] Does the source data need Spark-level flexibility, or is it already relational? [ ] Do you need multi-table transactions and T-SQL DML, or can table changes be implemented through Spark and Delta operations? [ ] Could a hybrid (Lakehouse → Warehouse) actually be the real answer? No data to test with yet? A quick way to practice both patterns above is to grab a free sample dataset (or a ready-made dashboard layout to reverse-engineer) from a site like Docynx. Over to you I'd love to hear how your team decided: Did you go Lakehouse, Warehouse, or both? What tipped the decision — team skill set, data shape, or something else entirely? Drop your setup in the comments — especially if you've got a "we picked wrong and had to migrate" story, those are always the most useful ones.Stop Hosting Report Images Publicly Just to Show Them in Power BI
The problem- showing an image meant making it public Say you're building a product catalog report, and you want each row in a table to show a small product photo next to its name and price. Power BI has always supported this through an image URL field you give it a web address, and it displays the image. Simple enough, until you think about where that image actually has to live. For Power BI to load an image from a URL, that URL generally needs to be publicly reachable on the internet. Which means product photos, employee headshots, or internal logos things that often shouldn't be sitting on a public website at all - ended up hosted somewhere publicly accessible anyway, purely so a report could display them. For a genuinely public product catalog, that might be fine. But for an internal HR dashboard showing employee photos, or an internal parts catalog with proprietary product images, publishing those images to a public URL just to satisfy a formatting requirement was never a comfortable trade-off. The alternative some people used embedding images as base64 text directly in the data - avoided the public hosting problem but came with its own cost =base64-encoded images are large blocks of text, and packing many of them into a semantic model bloats its size and slows things down, especially at any real scale. What changed In the August 2026 update, Power BI added support for using image files stored in OneLake directly as an image source - anywhere Power BI already accepts an image URL. This includes the image visual itself, table and matrix cells, the card visual, and button and list slicers. OneLake is Fabric's underlying storage layer think of it as your organization's own secure file storage that lives inside your Fabric tenant, not on the public internet. The important shift here is that your images now stay inside that same secure boundary as the rest of your data. Nothing has to be exposed publicly, and nothing has to be crammed into your model as bloated encoded text. A real example Picture that same product catalog report, but this time for an internal parts catalog your company doesn't want publicly visible - only your own staff should see these product images. 1. Upload your product photos to a Lakehouse in your Fabric workspace, the same way you might upload any other file to OneLake storage. 2. In your semantic model, add a column that holds the OneLake path to each product's image, matched to the right row (similar to how you'd normally store a web image URL). 3. In your table or matrix visual, set that column's data category to Image URL, the same step you'd already take for a normal web-hosted image. 4. Power BI now loads each product's photo directly from OneLake, displayed right in the table - with no image ever needing to be publicly hosted, and no bloated encoded text sitting in your model. The report looks and behaves exactly the way it would with any other image URL - the only thing that's changed is where the image actually lives, and who can reach it. Where this helps the most - Internal catalogs and HR-style reports, where showing a photo matter for usability, but the image itself shouldn't be public - Organizations with strict data governance requirements, where "the image is on a public URL" was never going to pass an internal security review - Reports with a lot of images, where base64 embedding was making the model uncomfortably large What to keep in mind This requires your images to actually be organized in a Lakehouse within your Fabric workspace first, so it's a bit more setup than just pasting a public URL - worth it specifically when keeping images private and secure actually matters for your use case. For a genuinely public-facing report where the images are meant to be seen by anyone anyway, a normal public image URL still works fine and doesn't need this extra step. If you've been quietly uncomfortable about hosting internal images publicly just so a report could display them, this is worth switching to the next time you build a report that needs pictures. Thanks for reading! Connect with me on: LinkedIn | Data With Pankaj - YouTubeDataflow Gen2 vs Copy Job vs Pipeline: Choosing by Workload, Not by Habit
Every Fabric team has a default tool. People who came from Azure Data Factory build a pipeline for everything. Power BI people open Dataflow Gen2 for everything. Newer teams might put every table into a Copy Job because the wizard is quick. Each tool does its own job very well. Problems start when it gets used for another tool's job: pipelines full of hand-built watermark logic, dataflows used only to copy tables, and Copy Jobs expected to handle transformations they were never built for. This post gives you a simple way to pick the right tool based on what the workload needs. One sentence each Copy Job moves data from A to B, including incremental loads, with as little setup as possible. Dataflow Gen2 shapes data. It cleans, merges, reshapes and applies business rules using Power Query. Pipeline coordinates work. It runs steps in order, handles dependencies and failures, and ties everything together. If you remember only one thing, remember this: Copy Job moves, Dataflow shapes, Pipeline orchestrates. Start with three questions about the workload Ask these before you open any editor. Am I moving data or changing it? If the data arrives at the destination looking mostly like the source, with only column mapping or type changes, you are moving data. If you are joining, deduplicating, deriving columns or applying business logic, you are transforming it. How does the source change? A one-time or full reload is a different problem from an ongoing incremental sync. Incremental loads based on change data capture (CDC) are different again. How many steps depend on each other? A single load on a schedule is one thing. "Load these, then validate, then transform, then run a stored procedure, then alert someone if it fails" is a workflow. Your answers usually point clearly to one tool. Copy Job: when the job is data movement Copy Job is the newest of the three and the one most often overlooked by teams stuck in old habits. Microsoft's decision guide lists its main scenarios as incremental copy and replication (both watermark-based and native CDC), data lake and storage migration, medallion ingestion, and out-of-the-box multi-table copy. Microsoft Learn The incremental support is the main reason to use it. In a pipeline, the Copy activity handles incremental copy through pipeline expressions and control tables, and only with watermarks. That means you build and maintain the control table, the lookup, the parameterized query and the watermark update yourself. Copy Job does this for you. Microsoft Learn Choose Copy Job when: You need to ingest many tables from a database into a lakehouse or warehouse. You want an initial full load followed by incremental updates, and you don't want to build watermark logic. The source supports CDC and you want inserts and updates merged into the destination automatically. The destination is outside Fabric. Copy Job supports 40+ destination connectors. Microsoft Learn Microsoft's own example fits this well: an analyst who needs multi-table selection across regional SQL Server instances, a bulk initial load, and then CDC-based incremental merges picks Copy Job because it supports both watermark-based and native CDC incremental copying through a wizard, and automatically detects CDC-enabled tables. Microsoft Learn Don't choose Copy Job when: You need real transformation. Its transformation support is rated low, and that is intentional. Microsoft Learn The load is one step in a larger workflow with conditions, retries and downstream dependencies. That belongs in a pipeline. Dataflow Gen2: when the job is shaping data Dataflow Gen2 is Power Query running at Fabric scale. It is the right tool when the value lies in the transformation logic. It offers 170+ built-in connectors, 300+ transformation functions in a visual interface, and data profiling tools for checking data quality. Microsoft Learn A common complaint is that dataflows are slow for large volumes. That used to be a fair criticism, but Fabric has added several performance features aimed at specific workloads: Fast Copy is for direct, high-throughput copies from a supported source with no transformations. It uses the same backend as the pipeline Copy activity. microsoftmicrosoft Modern Evaluator helps when you are shaping data from connectors that don't fold, or only partly fold. microsoft Partitioned Compute is for large, partitioned or multi-file datasets that can be processed in parallel. microsoft Staging lets you land raw data first and transform it afterwards (ELT), so ingestion and transformation don't compete in one pass. microsoft Choose Dataflow Gen2 when: Business logic is the main work: cleansing, standardizing codes, merging sources, calculated columns. The people who own the logic know Power Query and would struggle to maintain Spark or SQL. You are combining files, APIs, SharePoint lists and databases into one clean dataset. You want visual data profiling while you build. Don't choose Dataflow Gen2 when: You are only copying tables. Fast Copy makes this workable, but Copy Job is simpler, gives you incremental loads without extra work, and has less to maintain. Your destination isn't supported. Dataflow Gen2 lists around 7+ destination connectors, compared with 40+ for the copy tools. Microsoft Learn The transformations are very complex or code-heavy. At that point a notebook is usually the better choice. Pipeline: when the job is coordination A pipeline is not mainly a data movement tool. It is the orchestrator. Microsoft describes it as low-code orchestration that groups several activities together to complete a task. The Copy activity inside it is powerful, and it remains a strong option for very large migrations. The guide describes it as the best low-code choice for moving petabytes of data into lakehouses and warehouses, either ad hoc or on a schedule. Microsoft LearnMicrosoft Learn The real reason to use a pipeline is the control flow: dependencies, branching, retries, failure handling, parameters, and calling other items. Choose a pipeline when: Several steps must run in a set order, where step B only runs if step A succeeds. You need logic such as If/Else, ForEach over a metadata list, or waiting on an external event. You are combining different item types, such as a Copy Job, a dataflow, a notebook, a stored procedure and a web call. You need error handling and notifications around the whole process. You are doing a large, custom migration where you want detailed control over the Copy activity. Microsoft's scenario for pipelines describes a workflow that runs stored procedures, calls web APIs, moves files and executes other pipelines. That is orchestration, not simple ingestion. Microsoft Learn Don't choose a pipeline when: It would contain a single activity that just runs one dataflow on a schedule. Dataflows can be scheduled on their own. You would be rebuilding incremental loading by hand when Copy Job already supports your source. Side-by-side Copy Job Dataflow Gen2 Pipeline Main job Move and replicate Transform and shape Orchestrate Incremental loads Built in (watermark and CDC) Possible, but not its strength Manual (watermark and control tables) Transformation Low High None itself (calls other items) Destinations 40+ ~7+ Depends on activities Authoring Wizard Power Query Visual canvas and expressions Best owner Data integrator, analyst Analyst, data engineer Data engineer Warning sign you picked wrong Adding transformation workarounds Dataflow has no transformation steps Pipeline has one activity Common mistakes that come from habit The "everything is a pipeline" team. Every source table gets a Lookup, a ForEach, a parameterized Copy activity and a stored procedure to update the watermark. It works, but you now maintain a small custom framework that Copy Job gives you ready-made. Keep the pipeline and let it call the simpler pieces. The "everything is a dataflow" team. Dataflows with forty queries that just select a table and load it. Move the raw ingestion to Copy Job and keep dataflows for the layer where logic actually happens. The "Copy Job does it all" team. Trying to handle business rules with column mappings, then adding SQL views downstream to fix what should have been transformed properly. Once logic appears, add a Dataflow Gen2 or a notebook. The single-activity pipeline. A pipeline whose only purpose is to run one item on a schedule adds a layer to monitor without adding control. Use the item's own schedule until you really need dependencies. The pattern that usually wins: use all three, each for its own job For a typical medallion architecture, the tools fit together naturally: Bronze with Copy Job. Ingest source tables with built-in incremental or CDC loads. No custom watermark logic. Silver and Gold with Dataflow Gen2 (or notebooks for heavy, code-first logic). Apply cleansing, conformance and business rules where they are visible and easy to maintain. Pipeline around everything. Run ingestion, then transformation only if ingestion succeeded, then refresh or post-processing steps, and send an alert if anything fails. Each tool does what it is best at, and each is simpler because it isn't doing another tool's job. A 30-second decision checklist No transformation, and data must stay in sync over time? → Copy Job One-off or very large custom migration needing fine control? → Copy activity in a pipeline Transformation logic is the main work, and owners know Power Query? → Dataflow Gen2 Complex, code-first transformation at scale? → Notebook Multiple steps, dependencies, branching or error handling? → Pipeline, calling the tools above Closing thought The right question isn't "which tool do we use?" It is "what does this workload need?" Movement, shaping and coordination are three different problems, and Fabric gives you a dedicated tool for each. Teams that choose by workload build less custom plumbing, find problems faster, and hand solutions over more easily, because each piece does one clear job. Next time you start a new load, answer the three questions first, then pick the tool.Power BI Smart Table Visual: Excel style Column Filtering, Dynamic Column Headers, Grouping Columns
For a smoother reading experience with clearer screenshots and easy code copying, I suggest [reading this article on my website](https://www.techietips.co.in/articles/powerbi-managed-6500measures-smartly). It’s free, ad-free, distraction-free, and dedicated solely to sharing knowledge. The Table is the most commonly used visual in Power BI. Anyone who has spent years in Excel expects to click a header and filter that column. They expect related columns to sit under a common heading. The native table visual does none of this. There are grid visuals in the Microsoft visual app source that do. Most developers never get to use them, for two reasons that have nothing to do with the features. The good ones are licensed per developers or user, and that cost has to be justified to someone. The free ones are usually not Microsoft certified, which means an uncertified third party script is running inside your report, and that is a fair thing for a security team to say no to. So I built one. It is called Smart Table, and I built it with Claude. This post walks through what it does. 1. Excel style column filtering Every column header carries a funnel button in its own divided cell. Click it and you get what you would expect from Excel: sort, a set of conditions appropriate to the column’s type, and a searchable checkbox list of that column’s values. The menu names the column it belongs to, counts what you have ticked, and stays open while you sort. The conditions follow the column type rather than offering one generic list: Sync: does the filter stay in the grid or reach the report Sync is on by default. Filters applied in the header are pushed to the model, so every other visual on the page responds, exactly as if the user had used a slicer. Turn Sync off: Filtering then narrows this grid only. Global search The toolbar has a search box with its own operator dropdown: contains, is exactly, starts with, ends with, does not contain. It searches across every column at once. Power BI: Smart Table Excel Style Column Filtering 2. Dynamic column headers, driven by a measure A column header name is normally a static label. Here it can be a DAX measure. Arrival Month is set to April, and the Key Metrics headers read Lead Time (Avg: 30.00), Room Nights (Avg: 1.00), ADR (Avg: 160.50). Change the month and they recompute. The header honors the filter context, including the filters applied inside the grid itself. Filter Property down to Resort Hotel with the funnel and the averages in the header follow. Power BI Smart Table: Measure Driven Column Headers (Dynamic Column names) - YouTube 3. Column groups, without any extra tables Four groups across eleven columns: Reservation, Channel, Room Type, Key Metrics. Each one is a spanning header above its columns with its own colour. The thing to notice is what is not involved. No disconnected table. No field parameters. No two table visuals stacked on top of each other. No shapes placed in the background: which means your column groups move to the right along with your cursor. Power BI Smart Table: Grouping columns in a table without using a static table and a matrix visual - YouTube Known limits All of these come down to one idea: some filtering reaches the model and some is local to the grid. 30,000 rows That is the Power BI cap for a table mapping with this data reduction algorithm. Header filters push a real model filter, so filtering down to a workable set works fine. Do not point it at an unfiltered fact table. Value lists cap at 2,000 distinct values per column**, built from the loaded rows. On a high cardinality column like a guest name, use the search box in the menu. Ends with, Does not end with, and blank selections filter the grid only.** Power BI’s advanced filter operators have `StartsWith` and `Contains` but no `EndsWith`, and a blank has no equivalent in a model side `In` list. The menu tells you when you pick one of these. Bookmarks restore the data but not the ticked checkboxes.** The model filter is persisted by Power BI, so the rows come back correctly. The header checkboxes just will not show as ticked. I wrote this to make one point: the gaps in the built-in visuals are not permanent. Build your own, and with vibe coding that is no longer a months-long project. I am not sharing the .pbiviz file. It is an experiment, not tested for production grade reports. Feel free to checkout my [website](https://www.techietips.co.in) and reach out to me if you are interested to do these experiments along with me, I would like to continue developing this visual and will write detailed blogs in features explaining its features and implementations. I hope you learned something new. Feel free to share your thoughts in the comments section. Happy Learning!!!Why Refresh the Whole Model When Only One Table Changed? Power BI Finally Lets You Choose
The problem: refreshing everything when you only needed a little Picture a semantic model with a few tables: a large Sales fact table that updates every day, a Products table that changes maybe once a month, and a Currency Exchange Rate table that barely changes at all. For as long as most of us have used Power BI, hitting "Refresh" meant refreshing the entire model - every table, every time, whether it actually needed it or not. Worse, a normal refresh always did two things at once: it checked whether anything about your data source's structure had changed (a schema sync), and then it reloaded all the actual data. Most of the time you only care about one of those. You just want fresh numbers in your Sales table - you don't need Power BI to re-check whether your Currency table's column structure changed, because it never does. But there was no way to say "just get me new data, skip the schema check," or "refresh only this one table." You got the whole thing, every time. This wasn't just an annoyance - for larger models, refreshing everything when you only needed a fraction of it meant longer waits, more load on your source systems, and refresh operations eating into your daily refresh limits for no good reason. The old way to get around this If you needed genuinely granular control before this update, your only real option was going through the XMLA endpoint or the REST API - writing code to say "refresh just this table" or "refresh just this partition." This worked, but it required Premium or Fabric capacity, technical setup, and honestly, a level of comfort with scripting that most report builders don't have or need for day-to-day work. For anyone just using the standard Power BI Service interface, there was no such option - it was full refresh or nothing. What changed In the August 2026 update, Power BI brought this kind of control directly into the Service UI, no scripting required. You now get two separate improvements working together: First, refresh type options. Instead of one generic "Refresh" button that always does a schema sync plus a data refresh, you can now choose exactly what you want: - Refresh schema and data - the original all-in-one behavior, still available when you genuinely need both - Sync schema only - checks and applies any structural changes from your data source, like new columns, without reloading all the data - Refresh data only - reloads fresh data without checking for schema changes at all Second, table-level refresh. Instead of applying that choice to your entire model, you can now apply it to just one specific table. A real example Back to our Sales, Products, and Currency Exchange Rate model. Say it's the middle of a regular business day, and you just want today's sales numbers updated - nothing else has changed. Open the semantic model in Model view within the Power BI Service. Instead of hitting the generic top-level Refresh button, find the Sales table specifically. Choose Refresh data only for just that table. Power BI reloads fresh data into Sales, and leaves Products and Currency Exchange Rate completely untouched - no unnecessary schema check, no reloading tables that didn't need it. What used to be a full-model operation is now a targeted, few-second action. Where this saves the most time - Large models with tables that update on very different schedules - a fast-changing fact table alongside slow-changing dimension tables that rarely need touching - Direct Lake models specifically, where a Lakehouse table might have gained new columns, you're not ready to bring into your model yet - you can refresh the data without pulling in that schema change - Anyone managing refresh limits or capacity usage, where cutting unnecessary schema syncs and full-model reloads adds up to real savings over dozens of refreshes a day What to keep in mind This is a Power BI Service feature, so it's something you'll do from the web interface managing a published semantic model, not from Power BI Desktop while you're building. It's also worth remembering that "Sync schema only" and "Refresh data only" are meant to be used thoughtfully - if you genuinely don't know whether a data source's structure has changed recently, doing a full "Refresh schema and data" occasionally is still the safer choice, especially before a report goes out to a client or stakeholder. If you've been refreshing an entire model out of habit when only one table actually needed it, this is worth checking out the next time you're in the Power BI Service. Thanks for reading! Connect with me on: LinkedIn | Data With Pankaj - YouTubeDP-700: Microsoft Fabric Data Engineer: What It Is & Why It Matters in 2026
2025 was a year of massive evolution for the Microsoft data ecosystem, and with the rise of Microsoft Fabric as the unified analytics platform, Microsoft also introduced a fresh role-based certification designed specifically for data professionals: the DP-700 exam.
AI is Only as Smart as Your Model
As AI becomes more natively embedded across every analytics experience, data teams are discovering a hard truth: AI outcomes depend on the quality of the data they are grounded on. Every AI transformation journey starts with your data, and as AI opens new ways for consumers to interact with that data and draw out meaningful insights, the strength of the data foundation matters more than ever. Through this blog, we'll explore the importance of a semantic layer and how to transform your existing semantic model to be optimized for AI-enabled analytics experiences.
