Forum Discussion

ajomat's avatar
ajomat
Frequent Visitor
2 years ago

Incremental loading from MySQL database to fabric Lakehouse using Dataflow

Hello everyone,
I have a query about performing an incremental loading from MySQL database to Fabric Lakehouse, but it seems like my queries are not foldable. I read from some other internet sources that without foldable queries incremental refresh is not possible and also query folding is not supported in MySQL. How can I get around this? Is there any alternative solution ? Thanks in advance...

4 Replies

  • Hello! What steps do you have applied in Power Query? Assuming your source supports query folding, it is likely that one of the steps you applied broke the query folding. You can try to right click on the steps in Power Query and see if View Native Query is greyed out. If it is greyed out on the first step (Source) then this trick won't work to identify which step is breaking your query folding. If it is not greyed out, keep checking each step - when you get to a step the is greyed out this is the step that broke your query folding. You will need to remove it.

    • ajomat's avatar
      ajomat
      Frequent Visitor

      Hi,

      Thank you for the response.
      It seems like my queries are not foldable at the source.

      I followed the same step in SQL server database and the query was folding at source (the source was not greyed out and was marked as green - Screenshot below). It worked there.

      But when I used MySQL database the source is greyed out and the incremental refresh is not working anymore. This is what I could see in the window (screenshot below).

      • Anonymous's avatar
        Anonymous
        Not applicable

        Hi audreygerred ,thanks for the quick reply, I'll add further。

        Hi ajomat ,

        Regarding your question, the documentation does not state that Mysql supports query folding. If query folding is missing, then incremental refresh loses its meaning. There is no specific solution for the moment, you can only move the data to a data source that supports query folding. For example, SQL Server.

        Query folding on native queries - Power Query | Microsoft Learn