Forum Discussion
Post ProdigySplit DataSource Refresh rate (Some daily, others weekly)
Hi icassiem,
Thank you for providing the explanation. I recognize the difficulty involved in balancing cost management with maintaining up-to-date KPIs.
Since Power BI does not support configuring separate refresh schedules for each data source within a single dataset, and using a date condition in your Azure Function call still results in execution (causing token expiration and additional costs), the following approach offers an effective solution:
Split the Dataset Based on Refresh Frequency
To gain more control over refresh timing and reduce Azure Function costs, consider splitting your data sources into two separate Power BI datasets. Dataset A can include Salesforce, Zendesk, and other sources that require a daily refresh, while Dataset B should contain only the NetSuite data fetched via the Azure Function, which you can schedule to refresh only on specific days (e.g., the 1st and 15th of each month). To implement this, move the NetSuite query into a new .pbix file and publish it as a standalone dataset. Then, in your main Power BI report, use Composite Models or DirectQuery for Power BI Datasets to bring in both datasets. This approach keeps your report fully functional while significantly reducing unnecessary Azure Function executions and giving you better control over refresh schedules and associated costs.
Thank you! We’re glad to have you in the Microsoft Fabric Community and feel free to drop by anytime you need help or want to share ideas.Regards,
CST Member.
Super Usericassiem You can modify your Power Query to conditionally call the Azure Function based on the day of the month. This way, the function will only execute when necessary. However, this approach will still trigger the scheduled refresh, which might not fully solve the cost issue.
m
let
currentDate = DateTime.LocalNow(),
currentDay = Date.Day(currentDate),
Source = if currentDay <= 15 then
Table.FromColumns({Lines.FromBinary(Web.Contents("AZURE FUNCTION"))})
else
null
in
Source
Instead of refreshing the data source daily, you can adjust the refresh schedule to run less frequently for the NetSuite data. For example, you can set it to refresh only on specific days of the month (e.g., the 1st and 15th). This can be done using the scheduling options in your data integration tool or service.
Post ProdigyThank You bhanu_gautam
I have included this day clause in my azure app function call from within powerquery, but like you mention the service daily refresh might still refresh the datasource
i cant split the schedule as my kpi scorecard other kpi data sources i cant split, one can only set all the data sources to refreshed and not certain
Any other ideas please?
Regards
- icassiem
Also if i put the day condition around the function call the sources fail as the token has then expired due to the non token refresh from azure function app
?