Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.
So that's my case.
We have a on-premises MySql connected to a dataflow with a Data Gateway and using incremental refresh.
The incremental refresh is set to archive last 10 years and refresh the last 2 months.
The MySql table has a DateTime Column and the data is updated once a week using a Python script that update or create the records in the table for the last week.
I've reviewed the database and I could see the new info in the database but after run the dataflow some records from the last week are missing.
So there's any way to look at this?
a Python script that update or create the records in the table for the last week.
That's not what incremental refresh is about. To emulate differential refresh you need to widen your hot update window (say, last 4 weeks instead of last week) and you need to plan for periodic full refreshes, also of older partitions.
a Python script that update or create the records in the table for the last week.
This is only trying to explain the general process, but the problem is in the incremental refresh, the script in python is just the way to feed the database