Forum Discussion
Helper IIIBI architecture with Azure Data Lake Storage Gen2
Hi all,
Currently I run PowerBi directly on the sources, but I need to add history and avoid big full loads query on our sources.
Can I setup below architecture?
1. Connect some tables from our current SQL server with Navision to Azure Data Lake Storage Gen2 (+other sources) without any data transformation
2. Setup historical data tables in Azure Data Lake Storage Gen2
3. Use PowerBi Desktop to connect to the Azure Data Lake Storage Gen2
4. Transform data in PowerBi Desktop and create one standard "golden" dataset for the company (with relations, measures etc.)
5. Deploy this dataset for single source of truth via PowerBi premium to other users in PowerBi/Excel
1 Reply
not sure if this is what you want but read this link https://community.powerbi.com/t5/Service/In-a-DataFlow-are-partitioned-Azure-DataLake-Gen2-Parquet-files/m-p/2499398#M159357
My needs were to have partitioned (year=xx/month=xx/day=xx) Azure DataLake Gen2 containers that could be incrementally updated using a Parquet data format. I also wanted to use DataFlows so that multiple reports could use the transformed data.
I am not exactly sure of your needs but I did a POC to determine what would work and found out that I needed to use something like serverless SQL in azure synapse Analytics using views. Guy in a Cube - YouTube is starting a series on using Azure DataLake Gen2 with power bi and they already have some videos out there. Hope this helps.
