Forum Discussion
Data Engineering Template
timeseriesIQR thank you for sharing your work with the community. Why it's an interesting and potentially useful approach using Excel with Microsoft Fabric for data cleansing would not be considered a suggested pattern. Fabric provides multiple native workloads for exactly such a purpose as part of an ETL/ELT process or as standalone analytical tools. In no particular order here is a recommended toolset for data cleansing in Microsoft Fabric:
- DataFlow Gen2 - a visual way of loading data from a data source, transforming data to a new shape and storing into a native Fabric storage such as Data Warehouse. Can be combined with other tools in data orchestration workflows.
- Spark Notebook - a code first workload for Spark engine that allows various data manipulations for all kinds of phases in ETL/ELT processes. Preferred way in Lakehouse-based architecture.
- SQL explorer - a T-SQL oriented environment that enables all kinds of data manipulations using T-SQL language. Preferred way in SQL-first data architectures.
- KQL Query Set - a time-series oriented environment native for working with time-series data stored in KQL databases. Can be used for DDL and DML operations.
If you are a beginner in Microsoft Fabric and trying to find your ways and migrate from convenient and familiar office-level data tools to a professioanal toolset of a unified enterprise-grade data platfrom I suggest you start educating yourself using specialized learning tracks on Microsoft Learn like this Course DP-600T00-A: Implement analytics solutions using Microsoft Fabric - Training | Microsoft Learn
I don't have to. I am a programmer.