Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Enhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.

Dataflows - allow enterprise support to add "staging" step to existing dataflows

As a data engineer responsible for our Enterprise data platform I would like to collaborate with business users using Data Flows. A problem we are facing is that a source system gets multiple request for more or less the same data since there is no central coordination. For some source systems this leads to serious issues e.g. instability As our enterprise support I want to take existing dataflows and introduce a staging step which gets the data from the source and stores it in a raw'er format in our data lake. This raw'er format can then be linked to all dependent dataflows. The main benefit is that the source will only get one request and from that point all subsequent step are in an area I can control much better (data lake)
Status: Needs Votes
Comments
nplima1
New Member
What is the advantage of the Datalake for your control, as opposed to having: 1 workspace with "Staging Dataflow" - with only the Data Engineers as authorised users 1 workspace with "Normal Dataflow" - This 2nd dataflow uses the one from Staging as the data source; your reporting builders are the authorised users in this workspace.
fbcideas_migusr
New Member
Status changed to: Needs Votes