Solved: Re: Refreshing reports linked to dataflows - what ...

ruthhacche · ‎07-06-2020

I am confused as to what the manual refresh button on a report in PowerBI service does.

I have a dataflow including data from SalesForce and via a gateway to SQL. I schedule a reresh on a dataflow every hour.

If I create a dataset (and report) from the dataflow and refresh it manually I understand that the SQL data will only be as up to date as the last scehduled refresh in the dataflow. But will the SalesForce data be refreshed into the dataflow or actually into SalesForce (not needing a gateway and seemingly being directly available)? And, more importantly, would that be any different if we did not derive the dataset from a dataflow but instead just brought salesforce directly into the dataset model?

Icey · ‎07-13-2020

Hi @ruthhacche ,

Using a dataflow - doing a manual refresh on a report using the dataflow will do nothing because the dataflow has not yet refreshed to anything since the last time this dataset/report was refreshed by its schedule? The manual refresh is only down to the level of the latest refreshed dataflow, not down to the level of the source data.

The statement is not entirely correct. For importing, "a manual refresh on a report" won't cause the refresh of the dataset. It will just get the newest data from dataset. When you refresh a dataset, it will get the newest data from data sources.

So, when you connect to a dataflow in Power BI Desktop to create a report, the data source is dataflow. So, the data comes from the dataflow and it won't get from SalesForce or SQL. What is the data source when you connect, and where does the data obtained when refreshing come from.

Not using a dataflow - if I had a dataset that was not attached to a dataflow, a manual refresh would be down to the level of the source data and i would not have to wait until the dataflow refreshed to get up to date data.

Yes.

I schedule a refresh on the dataflow - this updates the dataflow down to source via gateway for SQL and directly for SalesForce.

Not entirely correct. If you create some queries used both columns from SalesForce and SQL, the data refresh of SalesForce will via gateway, too.

Do I then also have to schedule a refresh on each dataset to run just after the dataflow refresh or is this done automatically because the dataset knows it comes from that dataflow?

You need to set a scheduled refresh for each dataset.

Best Regards,

Icey

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

QueryWhiz · ‎11-16-2023

Hi @ruthhacche In this case I think you could go without a dataflow, you can directly pull your SF data to PBI and then set an incremental refresh so your report will always be updated. As a workaround, maybe you can try to test your connection with a 3rd party connector. I've tried windsor.ai, supermetrics (which doesn't have PBI as a destination, so you first have to export to Google Sheets and then export to PBI) and funnel.io. I stayed with windsor because it is much cheaper so just to let you know other options. In case you wonder, to make the connection first search for the Salesforce connector in the data sources list:

After that, just grant access to your Salesforce account using your credentials, then on preview and destination page you will see a preview of your Salesforce fields:

There just select the fields you need. It is also compatible with custom fields and custom objects, so you'll be able to export them through windsor. Finally, just select PBI as your data destination and finally just copy and paste the url on PBI --> Get Data --> Web --> Paste the url.

Mariusz · ‎07-06-2020

Hi @ruthhacche

Not sure if I understand your question but.

If you use dataflows you will need to refresh dataflows first and dataset second, two refreshes, one for each artefact.
I often use dataflows as an alternative to a data warehouse, a place where I can do all ETL and store data so the actual data source refresh times are reduced compared to when you try to cram all the process into the dataset.

If you create a dataflow connecting into SalseForce it will pull the data into dataflows first at the time when the manual or scheduled refresh is executed.

If you need the refresh to happen more often, you can increase the refresh rate or skip dataflows and use Direct Query and Automatic Refresh if you are lucky enough to have Premium Capacity in your disposal.

If you think you could skip using dataflows create datasets directly on the data source and its alined with you long term data strategy then just go ahead and do it, quite often if you question something maybe you do not need it.

https://docs.microsoft.com/en-us/power-bi/create-reports/desktop-automatic-page-refresh

Best Regards,
Mariusz

If this post helps, then please consider Accepting it as the solution.

Please feel free to connect with me.
LinkedIn

ruthhacche · ‎07-06-2020

Thank you. Using dataflows seems cleaner and i want to see if I can make that work. I dont mind if the refresh is not as 'realtime' - I just need to understand it. (DQ is not going to work for us).

To clarify the question perhaps - is the following correct?:

Using a dataflow - doing a manual refresh on a report using the dataflow will do nothing because the dataflow has not yet refreshed to anything since the last time this dataset/report was refreshed by its schedule? The manual refresh is only down to the level of the latest refreshed dataflow, not down to the level of the source data.
Not using a dataflow - if I had a dataset that was not attached to a dataflow, a manual refresh would be down to the level of the source data and i would not have to wait until the dataflow refreshed to get up to date data.

And an additional question:

I schedule a refresh on the dataflow - this updates the dataflow down to source via gateway for SQL and directly for SalesForce.
Do I then also have to schedule a refresh on each dataset to run just after the dataflow refresh or is this done automatically because the dataset knows it comes from that dataflow?

Not sure that is any clearer - the holes in my knowledge are making it difficult to even frame the question.

Icey · ‎07-13-2020