Forum Discussion
Orchestrating dataflow & semantic model refresh, warehouse/lakehouse
Hi
This really depends (oh another it depends!!) on several factors - budget, time and experience.
My take is that Dataflows are fine (afaik these are M Code data transforms ported out of Power BI and into Fabric?) if you limited data modelling skills in upstream languages, namely SQL.
Are you asking about Architecture for a datawarehouse or how to just orcestrate dataflows and dataset* refreshes?
Regards
JB
* Yes I know they are now called Semantic Models but you'd have thought they'd have called them Semantic Models first then moved to the Marketing name of Data Sets. Although other technologies also use the term "semantic models" to capture the relationships between schema and process so perhap it makes sense.
Hi, thanks for the reply,
The dataflows mainly pull from Oracle DB, some with SQL, some with the PBI Service GUI/M Code.
The dataflows are fine that's true, but I cannot orchestrate a nice refresh chain.
My main question is how to orchestrate the refresh of it all (when dataflows complete, refresh the semantic model). I jumped straight to Lakehouse/Warehouse/Gen2 Dataflows, as afaik, that's the only way to do the refresh orchestration properly (there is also Power Automate that I cannot use).
So I believe the best option may be transforming the Dataflows to Gen2 Dataflows, creating Lakehouse that gets data from those, create custom semantic model from this Lakehouse, do all the things I normally do in semantic model there and it will basically be the same solution I have now but allowing me to somehow refresh them all at once?