clrothgery's avatar
clrothgery
New Member
10 months ago
Status:
Need Clarification

Gen2 dataflows background refresh

Please give the option to turn on/off the background refresh to the database when building/editing Gen2 dataflows.  This is sending constant queries to the database (SAP) resulting in poor database performance.

1 Comment

  • miguel's avatar
    miguel
    Community Admin
    Status changed:
    New
    to
    Need Clarification

    In order to compute each step, we require to send the request to your source. Turning it off would mean that we wouldn't be able to communicate with your data source and validate that the request is valid. Imagine a scenario in which you manually type the name of a value which is incorrect and we aren't able to filter your table to say that there are no values with that string in your column. This wouldn't be possible to compute solely on the data preview as the preview typically only holds around 200 rows unless you start scrolling further down.

    If perhaps the idea is more geared towards creating a local cache of your data and only working against that cached and sampled dataset, definitely feel free to edit your idea to fit such description.

     

    We recommend using either one of the two features below to reduce the number of requests as well as the load to your data source:

    Using Schema view (Preview) - Power Query | Microsoft Learn - this makes sure that you only work with your table schema and are not loading anything other than just the schema

    Preview only step in Dataflow Gen2 (Preview) - Microsoft Fabric | Microsoft Learn - you can set a filter against your data to, fundamentally, work on a sample of your while authoring your dataflow without ever impacting the end-result which would go against the unfiltered dataset.

     

     

Recent ideas