Forum Discussion
Dataflow in pipeline succeeds but does nothing
- 1 year ago
I figured out what the issue was: it's the general unclear behavior with Dataflows with closing vs publishing.
When you exit out of the Dataflow editor, you think it's saving your latest changes because when you go back you will see the most recent state. But in fact it saved the previous state, and you can't save a Dataflow without running it.
This means that if you are doing a fetch+append data on a weekly or monthly basis, which is what I am doing, you have to wait until the last day of your time window to publish your Dataflow, and only then can you enable the auto-run schedule.
I figured out what the issue was: it's the general unclear behavior with Dataflows with closing vs publishing.
When you exit out of the Dataflow editor, you think it's saving your latest changes because when you go back you will see the most recent state. But in fact it saved the previous state, and you can't save a Dataflow without running it.
This means that if you are doing a fetch+append data on a weekly or monthly basis, which is what I am doing, you have to wait until the last day of your time window to publish your Dataflow, and only then can you enable the auto-run schedule.