Forum Discussion
Incremental refresh failed in Fabric
Hello, after having a problem due to consuming all the Fabric capacity because of having Dataflows2 without incremental refresh.
So, to solve it, we have activated incremental refresh on all the tables with the following parameters:
Once this is done in each table and using overwrite as the data destination:
Nos aparece el siguiente error:
The problem is not specifically the salesline table, since if I remove the incremental update in this table, the error appears in any of the other tables that have the incremental update.
If you can give me some help, I would appreciate it!
Hi data_mp_97
Thank you for reaching out to the Microsoft Fabric Forum Community.
Just make sure the incremental refresh date filter is added right after the Source step, before you do anything else like joins, custom columns, grouping, or other transformations. If your query needs a lot of complex shaping, it’s better to use a staging dataflow that only pulls data and applies the date filter, and then handle the rest of the transformations in a second dataflow.If I misunderstand your needs or you still have problems on it, please feel free to let us know.
Thanks.
2 Replies
Hi data_mp_97
Thank you for reaching out to the Microsoft Fabric Forum Community.
Just make sure the incremental refresh date filter is added right after the Source step, before you do anything else like joins, custom columns, grouping, or other transformations. If your query needs a lot of complex shaping, it’s better to use a staging dataflow that only pulls data and applies the date filter, and then handle the rest of the transformations in a second dataflow.If I misunderstand your needs or you still have problems on it, please feel free to let us know.
Thanks.Hi data_mp_97
Thank you for reaching out to the Microsoft Fabric Forum Community.
I hope the information provided was helpful. If you still have questions, please don't hesitate to reach out to the community.