fbcideas_migusr
7 years agoNew Member
Status:
Planned
Incremental Power BI Dataset refresh using Power BI Dataflow as a source
Incremental refresh only works for data sources where query folding is taking place. To my understanding Power BI Dataflow is not one of them. As I would like to offload both the actual data source as well as the Premium instance, I would like incremential refresh to work in both the Power BI Dataflow as well as in Power BI Dataset. If not, I need to choose. Rational: To be able to build an efficient ETL/ELT flow I need to be able to utilize incremental refresh both at the staging layer as well as in the data transformation and data model load layer. Workaround: If I instead of using Power BI Dataflow uses a SQL DB as a middle layer data store, I can do incremental refresh to the SQL DB and then also use incremental refresh from the SQL DB to the Power BI dataset.
5 Comments
- sandro_pedrettiNew MemberWe are using an AWS Redshift DWH with the following Data Refresh stages/workflow: AWS Redshift DWH => Dataflow (data ingest) => Datasets (BI-Model) We were planning to use Dataflows as an abstraction layer and ingest mechanism as proposed by Microsoft so that this ingested data can be used by the depending (Shared-) Datasets and thereby to implement a modern Self-Service BI solution. Because of the size of the data and the limited time available during nightly data refesh we require an incremental data refresh on all stages of the dataload (Dataflows & Datasets) to provide our users with updated date every day at 07:00 latest. So we need an Data Refresch workflow like this: AWS Redshift DWH => Incremental Refresh on Dataflows => Incremental Refresh on depending Datasets because we would like to use Power BI (Service) to query/anlayse with refreshed data from the DWH. So the incremental data refreshs have to work in tandem - currently it appears that this (incremental refreshs for both Dataflows and depending Datasets) is not working (yet) and we still have no official confirmation from Microsoft if this will be working/implemented in the future. If this is not working/or not designed this way we don't see that Power BI can be used in larger Enterprise environments...
- nishalitNew Member
Planned to release later this year.
- ptacquet1New Member
Necessary to use dataflows at enterprise level !!!
- giannis_pegiosNew Member
how long is this going to stay as "Planned"??
Whats the point of having incremental refresh on the dataflow and then having a dataset that fully refreshes every time?
- fbcideas_migusrNew MemberStatus added:Planned
Recent ideas
Control Schedule Activation per Environment in Deployment Pipelines
Currently, with the new multiple scheduler feature, we can define multiple run schedules for a Fabric item. However, when using deployment pipelines, there’s no way to control which schedules are act...frithjof_v6 minutes agoCommunity ChampionPlanned5.7KViews65likes12CommentsFabric Mirroring for Oracle fails on unconstrained NUMBER columns
Hi, might there be a way to Mirror Oracle database to Fabric when tabels include unconstrained NUMBER columns by setting preconditions for those as it common for Oracle databases? BR, MagnusMagnusBerglund13 hours agoNew MemberNew20Views0likes0CommentsEmbedded Power BI: allow 12/24-hour clock setting independent of formatLocale
Problem In Power BI embedded reports (app-owns-data, JavaScript SDK), localeSettings.formatLocale is the only way to influence date and time display. The hour cycle (12 or 24 hour) is tied to th...VijaySama5881 day agoNew MemberNew6Views0likes0CommentsAdd Initial Horizontal Scroll Position to Matrix and Table Visuals
Please add an option to Power BI Matrix and Table visuals that allows report developers to configure the initial horizontal scroll position when the visual loads. Suggested setting: Format visual >...Panteras011 day agoFrequent VisitorNew3Views0likes0CommentsFabric Mirroring (and all artifacts) - Reassign owner
Probelm: I have users who have created Mirroring from Azure SQL DB's (although this problem exists with all mirroring) and if they dont log into fabric often the mirroring breaks. While best prat...Jreed_74741 day agoNew MemberNew120Views1like1Comment