Forum Discussion

M98's avatar
M98
New Member
10 months ago
Solved

Dataflow Gen2 Incremental Load

I’m performing an incremental load in a Dataflow Gen2 using the LastModifiedDate column from the source table. This column is stored in Unix epoch format, so I convert it to a full datetime using thi...
capacities
workspaces
  • mmmmm_98's avatar
    mmmmm_98
    10 months ago

    The process has already been folded, but I suspect the SQL crash could also be due to concurrent access from other applications connected to the same SQL server. The additional load might have contributed to the failure.

mmmmm_98's avatar
mmmmm_98
Frequent Visitor
10 months ago

The process has already been folded, but I suspect the SQL crash could also be due to concurrent access from other applications connected to the same SQL server. The additional load might have contributed to the failure.

v-ssriganesh's avatar
v-ssriganesh
Community Support
10 months ago

Hi mmmmm_98,

Thanks for the update. If the process is already folded, that’s a good sign query folding is working as intended. Your suspicion about concurrent access from other applications adding load to the SQL Server is plausible and could definitely contribute to the crash, especially if the server is already under strain. To mitigate this, consider scheduling your Dataflow run during a low-traffic period or coordinating with other app owners to reduce simultaneous load. You could also check SQL Server logs for resource usage spikes during the failure to confirm. 

Best regards,
Ganesh Singamshetty

  • v-ssriganesh's avatar
    v-ssriganesh
    Community Support
    9 months ago

    Hello M98, 

    Could you please confirm if your query has been resolved by the provided solutions? This would be helpful for other members who may encounter similar issues.

     

    Thank you for being part of the Microsoft Fabric Community.