Forum Discussion
single dataset refresh, we observe multiple SQL statements
Hi manoj_0911 -Yes, this is expected behavior. To add to oussamahaimoud's explanation, Power BI can generate multiple SQL statements during a single refresh cycle because it performs several operations behind the scenes, including schema validation, partition evaluation, change detection, and data retrieval. The exact number of queries may vary depending on the incremental refresh policy, number of partitions, and whether query folding is occurring correctly.
If you're investigating performance, I'd recommend reviewing the actual partition refresh queries and confirming that the RangeStart/RangeEnd filters are being folded to the source. This will help ensure the incremental refresh is operating as efficiently as intended.
Hope this helps.
Hi manoj_0911,
Hope you're doing well!
Yes, this is completely expected behavior. Here's what each query type represents:
MAX(UPDATE_DATETIME) : this is the Detect Data Changes query. Before deciding which partitions to refresh, Power BI queries the max value of your change-detection column to determine if anything changed since the last refresh. If nothing changed, partition refreshes are skipped entirely.
TOP 1000: this is Power BI executing the actual incremental refresh partition queries. Each partition (both the incremental range and any historical partitions that qualify) gets its own filtered SQL query. The TOP 1000 you see in traces is typically the query folding preview/validation step, Power BI tests that the M query folds correctly to SQL before executing the full partition load. Note that the actual data load queries won't have TOP 1000, those will have full date-range predicates (WHERE RangeStart <= ... AND ... < RangeEnd). If you're only seeing TOP 1000, check whether you're looking at the validation phase vs. the actual partition refresh.
Metadata queries: Power BI queries schema/catalog information against the source (column types, table existence, etc.) as part of query planning. This is standard behavior for any dataset refresh, not specific to incremental refresh.
Hope this helps! Don't forget to accept as solution and give kudos in order to keep helping others.
Best regards,
Oussama (Data Consultant - Expert Fabric & Power BI)
Hi manoj_0911 ,
Thank you for reaching out to the Microsoft Community Forum. could you please try the proposed solution shared by oussamahaimoud ? Let us know if you’re still facing the same issue we’ll be happy to assist you further.
Regards,
Dinesh