Forum Discussion
incremental refresh on data lake
Hello,
you can implement incremental refresh in Power BI with both ADLS and AWS S3 as sources, the important part is having a proper date/time column and query folding where possible. With ADLS it’s generally smoother through Dataflows or Synapse, with S3 you may need an extra layer like Athena or Parquet files to make it efficientBest regards,
Daniele
powerbiexpert22
Yes, you can implement incremental refresh using lakehouse tables but your table must have datetime column.
Power BI can apply queryfolding to push the date filter back to source.
You must define RangeStart & RangeEnd Parameters.
DataSources Supported:
- ADLS Gen2 + Parquet/Delta.
- AWS S3 + Parquet/CSV.
Standard Step by Step guide how to set up incremental refresh in Power BI.
Step 1
Ensure your table contains a Date/Time column like CreatedDate or ModifiedDate.
Step 2
Connect Microsoft Power BI to ADLS/S3 data using:
Fabric Lakehouse
Databricks
Synapse
Athena/Snowflake
Step 3
Open:
Transform Data
in Power BI Desktop.
Step 4
Create two parameters:
RangeStart
RangeEnd
Type: Date/Time
Step 5
Filter the date column using:
[CreatedDate] >= RangeStart
And [CreatedDate] < RangeEnd
Step 6
Verify Query Folding by checking:
View Native Query
Step 7
Click:
Close & Apply
Step 8
In Model View, right-click the table → select:
Incremental Refresh
Configure:
Store data period
Refresh period
Step 9
Publish the dataset to Microsoft Power BI Service.
Step 10
Run the first refresh.
After that, Power BI refreshes only new or changed data incrementally.
If this helps, ✓ Mark as Kudos | Help Others