Forum Discussion
Azure Cost Management scheduled refresh
Hi all. I get data from Azure Cost management, but looking back over 12 months takes a huge amount fo time to refresh so when I try to set up a scheduled refresh it times out (I can't set up under a premium workspace as end users do not have that licence).
I have tried to set up an incremental to only update the last 5 days but I still get a time out error. Any advice!?
1) When I set up the scheduled refresh and the cloud connection, does the time period have to match what is in my original desktop query?
2) Any other ideas as to why the incremental refresh might be failing?
Knelly037 if the data is fully imported into PBI tables I would assume no further issues joining them.
But if so then with the limitation of not having access to a capacity or Fabric items the only additional idea I have is storing the data in a dedicated database and load the data from there to Power BI.
Best regards!
5 Replies
Hi Knelly037,
The Azure Cost Management connector is notoriously slow for large date ranges. Here are some solutions:
For your specific questions:
No, the time periods don't need to match. Your Desktop query should use the RangeStart/RangeEnd parameters for incremental refresh. The service will automatically adjust the date range based on your incremental refresh policy.
Common incremental refresh issues:
- Ensure you've set up RangeStart and RangeEnd parameters (DateTime type) in Power Query
- Filter your date column using these parameters: Table.SelectRows(Source, each [Date] >= RangeStart and [Date] < RangeEnd)
- Apply the incremental refresh policy in Desktop before publishing
- The initial refresh in the service will still take time to load historical data
Additional tips:
- Use aggregation: If you don't need daily granularity for all historical data, aggregate older data to monthly summaries
- Split the dataset: Consider separate reports for historical (Import, refreshed monthly) vs. recent data (refreshed daily)
- Dataflows: Pre-process the data in a dataflow with incremental refresh, then connect your report to the dataflow
Best regards!
PS: If you find this post helpful consider leaving kudos or mark it as solution
thanks Mauro89 appreciate the guidance.
1. Understood
2. I've followed all the steps, incremental refresh set for 7 days, and still get the timeout issue so I'm assuming just looking from outside there shouldn't really be any reason why it shouldn't work if those steps have been put in?
Splitting was something I looked at, e.g. run a manual refresh for historic data and seperate query just for the current month. The current month refreshes fine in isolation but if I then have a append step in desktop to join the two data sets together then I get the same timeout issue (I assume again because of size!?)
Knelly037 if the data is fully imported into PBI tables I would assume no further issues joining them.
But if so then with the limitation of not having access to a capacity or Fabric items the only additional idea I have is storing the data in a dedicated database and load the data from there to Power BI.
Best regards!
Hi Knelly037,
We would like to confirm if our community members answer resolves your query or if you need further help. If you still have any questions or need more support, please feel free to let us know. We are happy to help you.
Thank you for your patience and look forward to hearing from you.
Best Regards,
Prashanth Are
MS Fabric community supportHi @Knelly037,
We would like to confirm if our community members answer resolves your query or if you need further help. If you still have any questions or need more support, please feel free to let us know. We are happy to help you.
Thank you for your patience and look forward to hearing from you.
Best Regards,
Prashanth Are
MS Fabric community support
