Forum Discussion
Architecture set up using azure blob storage - advise needed
- 1 year ago
Hi jaryszek ,
Thank you for reaching out to Microsoft Fabric Community.
Switching to Azure Blob Storage, Dataflow Gen2, and Lakehouse is a smart move if you're aiming for a more reliable, scalable, and high-performance data pipeline into Power BI.
Instead of using Power BI’s Web.Contents connector to pull data directly from GitHub-which is often slow, fragile, and doesn’t refresh well in the Power BI Service-you can automate the process using GitHub Actions. These actions push your data into Azure Blob Storage, which acts as your raw data staging area.
From there, Dataflow Gen2 in Microsoft Fabric takes over. It ingests and transforms the data, loading it into a Lakehouse. This step lets you clean and optimize the data before Power BI even touches it.
Once your Lakehouse is ready, Power BI connects to it using DirectLake mode. This means no more waiting for data refreshes-Power BI reads the data directly from OneLake in-memory, giving you near-instant performance. Plus, you can reuse the same clean tables across multiple reports, which saves time and keeps everything consistent.
In short, this setup gives you an enterprise-grade architecture:
GitHub is your source of truth and automation engine.
Blob Storage is your raw data landing zone.
Dataflow Gen2 handles the ETL (Extract, Transform, Load).
Lakehouse becomes your central data model.
Power BI focuses purely on visualization and insights.It’s a clean, efficient, and scalable approach that makes managing and sharing data much easier in the long run.
If this post helps, then please consider Accepting as solution to help the other members find it more quickly.
Thank you.
thank you