Inconsistent Dataset Refresh Times

[Anonymous]

Good morning,

We are experiencing inconsistent Dataset Refresh durations for a particular dataset.  WWhen refreshed On Demand, the refresh takes less than 10 minutes, when scheduled it occasionally takes more than one hour.  The source is an Azure SQL Server with consistent load, with the only spikes occurring at 5 past the hour, every hour, and lasting roughly 10-15 minutes each time, when data is inserted into the Azure Database.  Having checked the Database logs, it appears as though the queries do not start hitting the database until well after the scheduled start of the Dataset Refresh process.

Scheduled	05/12/2020, 08:30:18	05/12/2020, 09:15:24	Completed
Scheduled	05/12/2020, 07:31:13	05/12/2020, 08:02:59	Completed
Scheduled	04/12/2020, 20:30:09	04/12/2020, 20:38:44	Completed
Scheduled	04/12/2020, 20:01:02	04/12/2020, 20:13:18	Completed
Scheduled	04/12/2020, 19:30:09	04/12/2020, 19:39:12	Completed
On demand	04/12/2020, 11:51:27	04/12/2020, 11:54:20	Completed
Scheduled	04/12/2020, 08:31:02	04/12/2020, 08:43:07	Completed
Scheduled	04/12/2020, 08:03:19	04/12/2020, 08:20:21	Completed
Scheduled	04/12/2020, 07:31:02	04/12/2020, 07:43:13	Completed
Scheduled	03/12/2020, 19:30:11	03/12/2020, 21:01:13	Completed
On demand	03/12/2020, 11:13:42	03/12/2020, 11:17:07	Completed
On demand	03/12/2020, 11:00:10	03/12/2020, 11:08:25	Completed
On demand	03/12/2020, 10:51:55	03/12/2020, 10:52:36	Completed
On demand	03/12/2020, 10:50:38	03/12/2020, 10:51:55	Completed
On demand	03/12/2020, 10:32:08	03/12/2020, 10:50:35	Completed
Scheduled	03/12/2020, 08:31:02	03/12/2020, 08:42:47	Completed
Scheduled	03/12/2020, 08:03:11	03/12/2020, 08:19:57	Completed
Scheduled	03/12/2020, 07:31:01	03/12/2020, 07:41:21	Completed

What is concerning is that these long durations appear to be randomly occurring, and as a result we cannot Subscribe to reports or dashboard which use this dataset and expect consistent results.

Has anybody encountered this before?  Is it possible to find some audit logs for the dataset refresh which would indicate the timestamps for each step of the dataset refresh process?  Thanks!

[mussaenda]

Hi @Anonymous ,

do you also have other reports refreshing with the same time of the report you mentioned above?

[Anonymous]

Hi @mussaenda , thanks for your reply.  We don't at that time, it is set up to be the only dataset refreshing at that time.  Would having multiple datasets refreshing close to each other affect the consistency of triggering the dataset refresh?  Thanks!

[mussaenda]

Hi @Anonymous,

Yes, I believe so. I also experienced your situation and what we did, we changed the timing of the reports refresh to avoid refreshing all at the same time. That quite helped the performance of the refresh.

Try also these suggestions:

Refresh