Finding Data Conversion Errors in Large Dataset

[C_PriceOH]

Hello!

I'm running into an issue with a larger chunk of data I'm working with in PBI's Power Query (~23 million records). I'm trying to set up an incremental refresh on a date/time field, but I'm running the following error (see below). This isn't allowing to query to refresh, so I can't identify the specific records that are causing the issue. Is anyone aware of a PBI setting or function I could use to force Power BI to complete the refresh and identify the errors, or another method? 

Thanks!

[C_PriceOH]

In the previous step to the filter step, there aren't any errors, but some records have Movement_Date values which are future dates. With my parameters set up like this below, could those future dates be causing the error with Power Query?

[lbendlin · Accepted solution]

the parameter values in Power Query have no meaning for incremental refresh. They only serve as a convenience for you , the developer, to limit the amount of data you use to develop the report.

The Power BI Service will overwrite these parameters with the partition boundaries based on the incremental refresh rules you specify.

Standard incremental refresh can NOT use dates in the future. If you need to do that you need to switch over to manual partition management. Advanced incremental refresh and real-time data with the XMLA endpoint in Power BI - Power BI | Micr... - you need to override the effectiveDate.

[lbendlin]

Before the filter step add a step to either remove the rows that have an error in that Movement_Date column or to replace the errors with a placeholder datetime that you can then check for in your report.  It will have to be in the past  (standard incremental refresh cannot handle future dates)  but not too far in the past so that your calendar table doesn't become bloated.