Forum Discussion
Community ChampionDataflow data
Hi smpa01,
Thank you for reaching out to Microsoft Fabric Community Forum.
SemPy is designed primarily for querying Power BI datasets. However, it is important to note that it does not have built-in capabilities for querying dataflows. For querying dataflows, you may need to consider alternative tools or methods, as suggested by the super users below.
Regards,
Vinay Pabbu
Super UserDataflows Gen1 are stored as Azure Blobs. It is technically possible, but very, very convoluted, to access them directly. Not worth the effort in my opinion.
Not really sure about Dataflows Gen2 - they give you the option to specify alternative destinations, so they may not even store anyting themselves.
smpa01lbendlin Thanks. But the method described here is not straightforward.
I ideally want to be able to read the data by making an API call or so in a notebook.
The fact that,
a. there is a way to read the data in notebook directly from some sources (databricks,ssas,smartsheet etc as API access is secured)
and
b. there would always be data residing in sources (sharepoint, sql, others - can only be read through dataflow as direct API access is restricted by the data owners)
if MS can provide access to dataflow data (for #b) using API (last time I checked it is currently not possible, PBI access token does not have Dataflow as audience), it would be so much more easier to utilize notebook as the single source to read data from different sources and build downstream applications.
The alternative is to scrape dataflow altogether and build ssas db (to utilize #a in notebook) but the caveat is dataflow usually has more memory attributed than dataset.
I don't to how to solve this problem. The ecosystem is following
previously power query (limited opportunites, not optimized),notebook ( query/ transform/ wrangle/ ML from different sources) -> downstream storage (Lakehouse,KQL, DW) ->downstream applications (BI, others)- lbendlin
Occam's Razor - challenge the need for the dataflow if you can do all this with Data Factory, Pipelines and notebooks.
Even in legacy Power BI dataflows are very often a solution looking for a problem.