Forum Discussion
Data Lakes and Power BI
Hope this is the right place to pose this question. I'm seeking advice on a current 'Build' I have been invited to use.
My company is extracting data via Python ETL from Three sources (Finance system, Jira and Netsuite), the extracts are fed (apolgies for non techncial language) into a SQL Server (SSMS), then into a "Cube". I am then provided access to Views they have built.
However... instead of extracting bulk data, the are extracting numerous (23) Queries that are preconfigured with formulas as they "Don't want loads of measures in PBI."
I am perplexed by this approach as it makes the modelling between datasets complex and there is duplication in the data fields.
Would a more efficient approach be to extract a general (catch all) data set from the systems, plonk it into a Server and allow PBI users to then access the data and build their own measures and functions?
EDIT: To add... by extracting multiple queries and then re-connecting them into views, doesn't that defeat the object.
1 Reply
- lbendlin
Super User
In an ideal world from the perspective of the "Citizen Data Scientist" you would not need any of these components like Python ETL, SQL Server, or cube (assuming SSAS OLAP?) . You would connect Power BI directly to the three data sources you mentioned.
In the real world there is the question of data governance, data sovereignity, DLP, "one version of the truth" etc etc.
Ask the architects of the current solution about their reasoning, when they last reviewed the design, and if they are aware of applications like Power BI.
